Validation step fails

[Mario Voigt]

Hey,

i installed an mxisd Server with version 1.3.1. It seems to run, but not completely. I am able to create new users in Synapse and invite them by email. But i also followed the "Validate" step in documentation:
https://github.com/kamax-matrix/mxisd/blob/master/docs/getting-started.md#configure

I am stucking between step 3 and 4:

.lookup.strategy.RecursivePriorityLookupStrategy - Host Internal allowed for recursion: true
.backend.sql.SqlThreePidProvider - SQL lookup
.backend.sql.SqlThreePidProvider - SQL query: SELECT user_id AS uid FROM user_threepids WHERE medium = ? AND address = ?
.backend.sql.SqlThreePidProvider - No match found in SQL
.lookup.provider.DnsLookupProvider - Performing DNS lookup for [e-mail]
.lookup.provider.DnsLookupProvider - Domain name for [e-mail]: kamax.io
.matrix.IdentityServerUtils - kamax.io is not an URL, using as-is
.matrix.IdentityServerUtils - Discovering Identity Server for kamax.io
.matrix.IdentityServerUtils - Performing SRV lookup
.matrix.IdentityServerUtils - Lookup name: _matrix-identity._tcp.kamax.io
.matrix.IdentityServerUtils - Record: _matrix-identity._tcp.kamax.io.        3599        IN        SRV        0 5 443 matrix.kamax.io.
.matrix.IdentityServerUtils - Found Identity Server for domain kamax.io at https://matrix.kamax.io:443
.lookup.provider.RemoteIdentityServerFetcher - Looking up email 3PID [e-mail] using https://matrix.kamax.io:443
.lookup.strategy.RecursivePriorityLookupStrategy - Found 3PID mapping: {medium: 'email', address: '[e-mail]', mxid: '@mxisd-lookup-test:kamax.io'}
.invitation.InvitationManager - Mapping for email:[e-mail] already exists, refusing to store invite
.http.undertow.handler.BasicHttpHandler - Request POST http://matrix.fablabchemnitz.de/_matrix/identity/api/v1/store-invite - Error M_ALREADY_EXISTS: A mapping already exists for t

Does anyone have an idea how i could deal to fix it? I am really new to the Matrix-World. I am using latest Synapse release 0.99.2, latest mxisd, latest Riot client.

kind regards and thanks, Mario

[Max]

@vmario89 This is the typical issue when the IP seen by mxisd when synapse tries to contact it is not a private IP. This tends to happen if your reverse proxy is using its public IP. Apache and nginx have configuration directives where you can enforce a specific IP/interface to be used.

If this is not desirable in your case, we can whitelist your IP, but that should only be done if there isn't any other way.

[Mario Voigt]

Hi @maxidorius,
i tried a lot of things with reverse proxy (found some mistake) and mxisd but no real difference. Im out of ideas how i could setup another way :-( The error is always the same.

the proxy config looks like:

<VirtualHost PUBLIC_IP:7080 127.0.0.1:7080>
     ServerName "matrix.fablabchemnitz.de"
     RedirectPermanent / "https://matrix.fablabchemnitz.de/"
 </VirtualHost>
 <VirtualHost PUBLIC_IP:7081 127.0.0.1:7081>
     ServerName "matrix.fablabchemnitz.de"
     ErrorLog ${APACHE_LOG_DIR}/error-matrix-synapse.log
     CustomLog ${APACHE_LOG_DIR}/access-matrix-synapse.log combined
     SSLEngine on
     SSLCertificateFile /etc/ssl/fablabchemnitz.de.pem
     ProxyPreserveHost on
     ProxyPass /_matrix/identity http://localhost:8090/_matrix/identity
     ProxyPass /_matrix http://localhost:8008/_matrix
     DocumentRoot /var/www/vhosts/matrix.fablabchemnitz.de
     Alias "/.well-known" "/var/www/vhosts/matrix.fablabchemnitz.de/.well-known"
     <Directory />
         Options Indexes
         AllowOverride None
         Require all granted
     </Directory>
 </VirtualHost>

my mxisd config looks like:

matrix:
   domain: 'matrix.fablabchemnitz.de'

 dns:
   overwrite:
     homeserver:
       client:
         - name: 'matrix.fablabchemnitz.de'
           value: 'http://127.0.0.1:8008'
       federation:
         - name: 'matrix.fablabchemnitz.de'
           value: 'http://127.0.0.1:8008'
 lookup:
   recursive:
     enabled: true
     bridge:
       enabled: true
       mappings:
         email: 'http://localhost:8091'

 invite:
   resolution:
     recursive: true

By the way the federation stuff looks good so far: https://matrix.org/federationtester/api/report?server_name=matrix.fablabchemnitz.de

Do you have a tip what i could do to change?

thanks for your reply and good night ;)

[Max]

@vmario89 My understanding of Apache is that specifying the IP in the <VirtualHost ....> part is what would make it use your public IP, as it's listed as the first entry. Normally, you would just put a wildcard and let Apache decide the best address to use.

For the sake of I understand what is really going on here, could you try with a * for now instead of your IPs, just to see if that works better?

[Mario Voigt]

Hi, i tried this but same situation. Maybe its a problem with nginx. I am running our server with Plesk. This runs nginx in front and uses apache for reverse proxy things. As my services are running for tests to make it work, i purged the synapse and mxisd databases to setup new ones.

The Output from Matrix Synapse is:

2019-03-05 10:24:35,390 - synapse.http.server - 112 - ERROR - POST-36- Failed handle request via 'RoomMembershipRestServlet': <XForwardedForRequest at 0x7fb40c01e710 method='POST' uri='/_matrix/client/r0/rooms/!UPuCcvxkRCRtznpatT:matrix.fablabchemnitz.de/invite' clientproto='HTTP/1.1' site=8008>
Traceback (most recent call last):
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/http/server.py", line 81, in wrapped_request_handler yield h(self, request)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/http/server.py", line 316, in _async_render callback_return = yield callback(request, **kwargs)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/rest/client/v1/room.py", line 669, in on_POST txn_id
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/handlers/room_member.py", line 706, in do_3pid_invite txn_id=txn_id
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/handlers/room_member.py", line 817, in _make_and_store_3pid_invite inviter_avatar_url=inviter_avatar_url
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/handlers/room_member.py", line 917, in _ask_id_server_for_third_party_invite invite_config
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks result = g.send(result)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/http/client.py", line 350, in post_urlencoded_get_json raise HttpResponseException(response.code, response.phrase, body) synapse.api.errors.HttpResponseException: 400: b'Bad Request'

and journalctl -u mxisd.service:

d.http.undertow.handler.identity.v1.SingleLookupHandler - Got single lookup request from 217.92.209.189 with client Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KH
d.lookup.strategy.RecursivePriorityLookupStrategy - Host 217.92.209.189 allowed for recursion: false
d.backend.sql.SqlThreePidProvider - SQL lookup
d.backend.sql.SqlThreePidProvider - SQL query: SELECT user_id AS uid FROM user_threepids WHERE medium = ? AND address = ?
2.c3p0.impl.AbstractPoolBackedDataSource - Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 2, acquireRetryAttempts -> 10, acquireRetryDela
d.backend.sql.SqlThreePidProvider - No match found in SQL
d.lookup.strategy.RecursivePriorityLookupStrategy - No 3PID mapping found
d.http.undertow.handler.identity.v1.SingleLookupHandler - No mapping was found, return empty JSON object
d.http.undertow.handler.identity.v1.SingleLookupHandler - Got single lookup request from 85.214.33.238 with client Synapse/0.99.2 - Is recursive? false
d.lookup.strategy.RecursivePriorityLookupStrategy - Host PUBLIC_IP allowed for recursion: false
d.backend.sql.SqlThreePidProvider - SQL lookup
d.backend.sql.SqlThreePidProvider - SQL query: SELECT user_id AS uid FROM user_threepids WHERE medium = ? AND address = ?
d.backend.sql.SqlThreePidProvider - No match found in SQL
d.lookup.strategy.RecursivePriorityLookupStrategy - No 3PID mapping found
d.http.undertow.handler.identity.v1.SingleLookupHandler - No mapping was found, return empty JSON object
d.invitation.InvitationManager - Handling invite for email:[e-mail] from @vmario89:matrix.fablabchemnitz.de in room !UPuCcvxkRCRtznpatT:matrix.fablabchemnitz.
d.lookup.strategy.RecursivePriorityLookupStrategy - Host Internal allowed for recursion: true
d.backend.sql.SqlThreePidProvider - SQL lookup
d.backend.sql.SqlThreePidProvider - SQL query: SELECT user_id AS uid FROM user_threepids WHERE medium = ? AND address = ?
d.backend.sql.SqlThreePidProvider - No match found in SQL
d.lookup.provider.DnsLookupProvider - Performing DNS lookup for [e-mail]
d.lookup.provider.DnsLookupProvider - Domain name for [e-mail]: kamax.io
d.matrix.IdentityServerUtils - kamax.io is not an URL, using as-is
d.matrix.IdentityServerUtils - Discovering Identity Server for kamax.io
d.matrix.IdentityServerUtils - Performing SRV lookup
d.matrix.IdentityServerUtils - Lookup name: _matrix-identity._tcp.kamax.io
d.matrix.IdentityServerUtils - Record: _matrix-identity._tcp.kamax.io.        3599        IN        SRV        0 5 443 matrix.kamax.io.
d.matrix.IdentityServerUtils - Found Identity Server for domain kamax.io at https://matrix.kamax.io:443
d.lookup.provider.RemoteIdentityServerFetcher - Looking up email 3PID [e-mail] using https://matrix.kamax.io:443
d.lookup.strategy.RecursivePriorityLookupStrategy - Found 3PID mapping: {medium: 'email', address: '[e-mail]', mxid: '@mxisd-lookup-test:kamax.io'}
d.invitation.InvitationManager - Mapping for email:[e-mail] already exists, refusing to store invite
d.http.undertow.handler.BasicHttpHandler - Request POST http://matrix.fablabchemnitz.de/_matrix/identity/api/v1/store-invite - Error M_ALREADY_EXISTS: A mapping already exists for

maybe it has to do with "X-Forwarded-For $remote_addr" within nginx?

regards, MArio

[Mario Voigt]

okay i checked nginx and as i can see the x-forwarded-for is already enabled:

 location / {
     proxy_pass https://85.214.33.238:7081;
     proxy_set_header Host             $host;
     proxy_set_header X-Real-IP        $remote_addr;
     proxy_set_header X-Forwarded-For  $proxy_add_x_forwarded_for;
     proxy_set_header X-Accel-Internal /internal-nginx-static-location;
     access_log off;
 }

mmhh

[Max]

Thank you for the detailed logs and they indicate that your public IP is used for internal coms, which triggers mxisd protections against being the source of an amplification attack due to doing recursive lookups (much like DNS).

If you cannot make it so your reverse proxy/synapse can talk to mxisd using an internal IP from a private subnet, then you'll need to disable the protection using the following in your mxisd.yaml file:

lookup:
  recursive:
    allowedCidr:
      - '127.0.0.0/8'
      - '10.0.0.0/8'
      - '172.16.0.0/12'
      - '192.168.0.0/16'
      - '::1/128'
      - 'fe80::/64'
      - '<YOUR IP HERE>/32'

[Mario Voigt]

Hey,
this helped to fix the issue. But i will try to change the settings to make it work with the correct interface instead later. regards and many thanks for your help. If i have solved the problem another way i will post here again.

regards, Mario

[Max]

Thank you for the feedback! I'll close this, but feel free to add to it later.