Errors when viewing Discovery settings in Element Android

[Xavier Johnson]

With the v2 API enabled, ma1sd hits a few errors when I try use Element Android to view the Discovery screen to review which emails I have marked as shared. Viewing the same information works just fine in Element Web.

It seems like Element Android might be sending some incorrect requests, but it's supposed to support the v2 API now.

(Background: I have two email addresses linked to my Matrix account, with only one of them being discoverable. I used Element Web some time ago to set them up, which worked properly.)

My ma1sd config is:

matrix:                                                                                                  
  domain: 'my_domain.tld'
  v1: true
  v2: true
  ## Remove default matrix-org server
  identity:
    servers:
      myOtherServers: []

server:
  name: 'id.my_domain.tld'

key:
  path: '/var/lib/ma1sd/keys'

storage:
  backend: postgresql
  provider:
    postgresql:
      database: '//localhost/ma1sd'
      username: 'username'
      password: 'password'
  
threepid:
  medium:
    email:
      identity:
        from: "noreply-id@my_domain.tld"
        name: "name"
      connectors:
        smtp:
          host: "localhost"
          tls: 0
          port: 25
  
hashing:
   enabled: true
   pepperLength: 20
   rotationPolicy: per_requests
   hashStorageType: in_memory
   algorithms:
    - sha256
   requests: 10 

synapseSql:
   enabled: true
   type: 'postgresql'
   connection: '//localhost/synapse?user=synapse_user&password=postgres'
   lookup:
     query: 'select user_id as mxid, medium, address from user_threepid_id_server'
   legacyRoomNames: false

logging:
  root: debug
  app: debug
  requests: true

Thanks to ma1sd's HTTP request logging, I was able to get a step-by-step log of what happens when viewing the Discovery screen, including possibly bad behaviour from Element Android:

1. Element Android sends two requests for GET /_matrix/identity/v2. I have two email addresses as my 3PIDs, so it's probably sending one request per address.
2. ma1sd responds to each request with 200 and an empty body (good)
3. Element Android sends two requests for GET /_matrix/identity/v2/account with an "Authorization: Bearer" header with a token that should have come from a response to POST /_matrix/identity/v2/account/register, but no request was made to it yet. It must be that Element Android saved a token it got from an old request made long ago that has since expired.
4. ma1sd responds to one of the requests with 401 and a body of {"errcode":"M_UNAUTHORIZED","error":"Supplied credentials are invalid","success":false}
5. For the other request, ma1sd hits the below exception, and responds with 500 and {"errcode":"M_UNKNOWN","error":"Unexpected row count after DB action: 0","success":false}:

[XNIO-1 task-12] ERROR io.kamax.mxisd.http.undertow.handler.SaneHandler - Unknown error when handling htt
p://localhost:8090/_matrix/identity/v2/account
java.lang.RuntimeException: Unexpected row count after DB action: 0
        at io.kamax.mxisd.storage.ormlite.OrmLiteSqlStorage.lambda$deleteToken$13(OrmLiteSqlStorage.java:
405)
        at io.kamax.mxisd.storage.ormlite.OrmLiteSqlStorage.withCatcher(OrmLiteSqlStorage.java:242)
        at io.kamax.mxisd.storage.ormlite.OrmLiteSqlStorage.deleteToken(OrmLiteSqlStorage.java:402)
        at io.kamax.mxisd.auth.AccountManager.deleteAccount(AccountManager.java:148)
        at io.kamax.mxisd.http.undertow.handler.AuthorizationHandler.handleRequest(AuthorizationHandler.j
ava:64)
        at io.kamax.mxisd.http.undertow.handler.SaneHandler.handleRequest(SaneHandler.java:71)
        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376)
        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
[XNIO-1 task-13] WARN io.kamax.mxisd.auth.AccountManager - Account not found.

6. Eventually, Element Android sends two requests of POST /_matrix/identity/v2/account/register, and ma1sd responds to each with a valid access token.
7. Element Android requests GET /_matrix/identity/v2/account using the returned token as the "Authorization: Bearer" token, and ma1sd responds to it successfully.
8. Element Android requests POST /_matrix/identity/v2/lookup, but sets the pepper to some value I've never seen before, since it never made a request to /_matrix/identity/v2/hash_details!
9. Naturally, ma1sd responds with 400 and {"errcode":"M_INVALID_PEPPER","success":false}.

[Xavier Johnson]

As I mentioned in element-hq/element-android#2531, the reason why Element Android doesn't ask for a new pepper is because ma1sd's error response doesn't specify "error" in its body, which is against the spec and is thus not recognized by Element Android as a valid missing-pepper error.

That means this should be fixable by updating ma1sd to add an error message in its invalid-pepper response (as well as all other error responses).

This might still not explain the "Unexpected row count after DB action: 0" error, though. I'll try to add the error message myself to see what happens.

[T3chTobi]

Why are you using query: 'select user_id as mxid, medium, address from user_threepid_id_server'?

I'm using query: 'select user_id as mxid, medium, address from user_threepids', because my table user_threepid_id_server is empty and the table user_threepids is filled with some data.

Is that how it should be in my case or should I be using your code?

[Xavier Johnson]

It's because of #27. user_threepids contains all 3PID mappings, whereas user_threepid_id_server contains only those that the user opted to share. Because of this, using user_threepids will cause ma1sd to treat all 3PIDs as shared, instead of only those the user opted to share.

[Xavier Johnson]

#77 should fix this, so this can be closed unless someone else hits a problem. Thanks for accepting the pull request @ma1uta :)