Cannot receive external email

Question

Cannot receive external email

MLaudren opened this issue 8 years ago · comments

Hi,

I was trying Peps, and I'm not able to receive any external email but can send them.
According to https://mxtoolbox.com/, Only the SMTP Banner is incorrect:
SMTP Banner Check Reverse DNS does not match SMTP Banner.

Others passed tests:
SMTP Reverse DNS Mismatch OK - 1.2.3.4 resolves to mydomain.ex
SMTP Valid Hostname OK - Reverse DNS is a valid Hostname
SMTP TLS OK - Supports TLS.
SMTP Connection Time 1.031 seconds - Good on Connection time
SMTP Open Relay OK - Not an open relay.
SMTP Transaction Time 3.344 seconds - Good on Transaction Time

If I look into the log on smtpout container, I can see that my email is receive but I have the following error, and nothing appear in the targeted account.
I try using gmail and outlook but both fail...

Error section :
[CRIT] [-] [core] TypeError: The header content contains invalid characters
[CRIT] [-] [core] at ClientRequest.OutgoingMessage.setHeader (http.js:733:13)
[CRIT] [-] [core] at new ClientRequest (http.js:1429:14)
[CRIT] [-] [core] at Object.exports.request (http.js:1899:10)
[CRIT] [-] [core] at Request.start (/usr/local/haraka/node_modules/request/index.js:603:30)
[CRIT] [-] [core] at Request.write (/usr/local/haraka/node_modules/request/index.js:1226:28)
[CRIT] [-] [core] at MessageStream.ondata (stream.js:51:26)
[CRIT] [-] [core] at MessageStream.emit (events.js:95:17)
[CRIT] [-] [core] at ChunkEmitter. (/usr/lib/node_modules/Haraka/messagestream.js:352:18)
[CRIT] [-] [core] at ChunkEmitter.emit (events.js:95:17)
[CRIT] [-] [core] at ChunkEmitter.end (/usr/lib/node_modules/Haraka/chunkemitter.js:92:14)

I have Haraka@2.7.3 but it's the same with the latest 2.8.0-alpha7

I can provide extensive log if needed.

Any help is welcomed.
Thanks!

MLaudren · Answer 1 · Fri Mar 25 2016 00:28:47 GMT+0800 (China Standard Time)

Here are more details following a discution with haraka.

The error come from smtpin plugin.

Here is plugin list ordered and full log of the error.

haraka.txt
haraka-full.txt

Khiem · Answer 2 · Thu Mar 31 2016 10:21:35 GMT+0800 (China Standard Time)

I've gotten the same error as well

BTR Naidu · Answer 3 · Wed Apr 27 2016 23:39:32 GMT+0800 (China Standard Time)

How were you able to trace detailed log out of peps_smtpout container?

BTR Naidu · Answer 4 · Wed Apr 27 2016 23:48:32 GMT+0800 (China Standard Time)

I too have below error. Anybody found any solution so far??

[DEBUG] [3320F997-38F3-4FE2-883C-7DEC91B71B83.1] [core] running data_post hook in smtpin plugin
[CRIT] [-] [core] TypeError: The header content contains invalid characters
[CRIT] [-] [core] at ClientRequest.OutgoingMessage.setHeader (http.js:733:13)
[CRIT] [-] [core] at new ClientRequest (http.js:1429:14)
[CRIT] [-] [core] at Object.exports.request (http.js:1899:10)
[CRIT] [-] [core] at Request.start (/usr/local/haraka/node_modules/request/index.js:603:30)
[CRIT] [-] [core] at Request.write (/usr/local/haraka/node_modules/request/index.js:1226:28)
[CRIT] [-] [core] at MessageStream.ondata (stream.js:51:26)
[CRIT] [-] [core] at MessageStream.emit (events.js:95:17)
[CRIT] [-] [core] at ChunkEmitter. (/usr/lib/node_modules/Haraka/messagestream.js:352:18)
[CRIT] [-] [core] at ChunkEmitter.emit (events.js:95:17)
[CRIT] [-] [core] at ChunkEmitter.end (/usr/lib/node_modules/Haraka/chunkemitter.js:92:14)
[NOTICE] [-] [core] Shutting down

Carlos Cárcamo · Answer 5 · Fri May 27 2016 13:58:17 GMT+0800 (China Standard Time)

I have the same problem :(

Sebastián Ramírez · Answer 6 · Tue Nov 29 2016 09:56:17 GMT+0800 (China Standard Time)

Guys, you might want to try my branch (while they review and merge my PRs). I fixed the inbound and outbount containers.

It's here: https://github.com/tiangolo/PEPS/tree/update-haraka-out

I'm using NodeJS official images as the base images (following Docker best practices) instead of the Phushion base image, so it should be less prone to errors now (and easier to debug).

If it works for you, it would be great if you can also report it in the PRs I created here so that they can merge the fixes:

#69

#71

MLaudren · Answer 7 · Tue Nov 29 2016 17:03:07 GMT+0800 (China Standard Time)

Installation in progress,
Will check if it's working.

Thanks!

MLaudren · Answer 8 · Tue Nov 29 2016 18:00:26 GMT+0800 (China Standard Time)

peps_server and peps_smtpin stop working about 30s after I start them.

I'll try to find some logs to see what happen.

MLaudren · Answer 9 · Tue Nov 29 2016 18:35:45 GMT+0800 (China Standard Time)

have this in peps_server
*** Running /etc/my_init.d/00_regen_ssh_host_keys.sh...
*** Running /etc/my_init.d/10peps.sh...
Invalid parameter for option --smtp-out-host, in Parameters. Syntax:
--smtp-out-host
File "lib/stdlib/core/args/server_arg.opa", line 250, characters 23-27, (250:23-250:27 | 9323-9327)
fail:
Error : uncaught OPA exn { 'size`': 2,
fail: '',
position: 'File "lib/stdlib/core/args/server_arg.opa", line 250, characters 23-27, (250:23-250:27 | 9323-9327)' }

/opa/usr/local/lib/opa/stdlib/opabsl.opp/opabslNodeJsPackage.js:935
throw new (global.Error)("Your toplevel contains value which can't be synchronously computed");}global.top0 = global.top1;return global.topval;};
^

Error: Your toplevel contains value which can't be synchronously computed
at global.BslCps_topwait (/opa/usr/local/lib/opa/stdlib/opabsl.opp/opabslNodeJsPackage.js:935:7)
at Object. (/peps/_build/com.mlstate.webmail.static.opx/main.js:234:33)
at Module._compile (module.js:410:26)
at Object.Module._extensions..js (module.js:417:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Module.require (module.js:354:17)
at require (internal/module.js:12:17)
at Object. (/peps/_build/com.mlstate.webmail.model.opx/main.js:11:1)
at Module._compile (module.js:410:26)
*** /etc/my_init.d/10peps.sh failed with status 1

*** Killing all processes...

and for smtpin:
loaded TLD files: 1=1515 2=6003 3=2319
loaded 8126 Public Suffixes
loglevel: LOGWARN
Starting up Haraka version 2.8.11
[CRIT] [-] [core] Error: could not get domain from peps:8999
[CRIT] [-] [core] at Request._callback (/usr/local/haraka/plugins/smtpin.js:43:31)
[CRIT] [-] [core] at self.callback (/usr/local/haraka/node_modules/request/request.js:186:22)
[CRIT] [-] [core] at emitOne (events.js:96:13)
[CRIT] [-] [core] at Request.emit (events.js:188:7)
[CRIT] [-] [core] at Request.onRequestError (/usr/local/haraka/node_modules/request/request.js:845:8)
[CRIT] [-] [core] at emitOne (events.js:96:13)
[CRIT] [-] [core] at ClientRequest.emit (events.js:188:7)
[CRIT] [-] [core] at Socket.socketErrorListener (_http_client.js:308:9)
[CRIT] [-] [core] at emitOne (events.js:96:13)
[CRIT] [-] [core] at Socket.emit (events.js:188:7)

which is 'normal' because it try to connect to peps_server...

Sebastián Ramírez · Answer 10 · Tue Nov 29 2016 19:57:31 GMT+0800 (China Standard Time)

@MLaudren It says there's an error in your toplevel, wich might refer to the top level domain. And in Haraka (SMTP in) it says it couldn't get the domain. Are you sure you updated the domain file with your domain?

To do that you need to enter the PEPS directory and run:

echo "example.com" > domain

...replacing example.com with your domain.

MLaudren · Answer 11 · Tue Nov 29 2016 20:28:44 GMT+0800 (China Standard Time)

Yep, the domain and host files are correct.

in the peps/peps.sh script, the --smtp-out-host refer to this variable: $SMTPOUT_PORT_25_TCP_ADDR
And I cannot find it in other config file...
Is this variable built at runtime? of should it be injected during the build/run process?

Sebastián Ramírez · Answer 12 · Tue Nov 29 2016 20:35:30 GMT+0800 (China Standard Time)

It's used here: https://github.com/tiangolo/PEPS/blob/update-haraka-out/Makefile#L59

And it's declared here: https://github.com/tiangolo/PEPS/blob/update-haraka-out/Makefile#L14

Then, are you using make to start everything?

MLaudren · Answer 13 · Tue Nov 29 2016 20:46:22 GMT+0800 (China Standard Time)

I do a make build and make run
but before everything I clean up with make rm

Did I miss something?

Edit:
After starting the peps_server container, I've connect to it and look at the env variables:

$ docker exec -it peps_server /bin/bash
root@localhost:/# env
SMTPOUT_PORT_465_TCP_PORT=465
HOSTNAME=localhost
SMTPOUT_PORT_465_TCP_ADDR=172.17.0.4
SMTPOUT_PORT_465_TCP=tcp://172.17.0.4:465
SMTPOUT_PORT_465_TCP_PROTO=tcp
MONGOD_PORT_27017_TCP_PROTO=tcp
SOLR_ENV_LC_ALL=en_US.UTF-8
SOLR_PORT_8983_TCP_PROTO=tcp
LS_COLORS=
SOLR_PORT_8983_TCP_ADDR=172.17.0.3
MONGOD_PORT_27017_TCP_ADDR=172.17.0.2
MONGOD_PORT_27017_TCP=tcp://172.17.0.2:27017
SOLR_ENV_LANGUAGE=en_US:en
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SMTPOUT_PORT=tcp://172.17.0.4:465
MONGOD_PORT=tcp://172.17.0.2:27017
SOLR_ENV_JAVA_HOME=/usr/lib/jvm/java-7-oracle
PWD=/
SOLR_NAME=/peps_server/solr
SOLR_PORT=tcp://172.17.0.3:8983
NODE_PATH=/opa/usr/local/lib/opa/static:/opa/usr/local/lib/opa/stdlib:/opa/usr/local/lib/opa/stdlib/stdlib.qmljs:/usr/local/lib/node_modules:node_modules::/peps/_build
MONGOD_PORT_27017_TCP_PORT=27017
SOLR_ENV_SOLR_DATA=/opt/solr/example/solr
SHLVL=1
HOME=/root
SOLR_ENV_SOLR_ROOT=/opt/solr
OPADIR=/opa/usr/local/lib/opa
SOLR_ENV_LANG=en_US.UTF-8
SMTPOUT_NAME=/peps_server/smtpout
SOLR_ENV_SOLR_VERSION=4.10.4
MONGOD_NAME=/peps_server/mongod
LESSOPEN=| /usr/bin/lesspipe %s
SOLR_ENV_SOLR=solr-4.10.4
SOLR_PORT_8983_TCP_PORT=8983
SOLR_PORT_8983_TCP=tcp://172.17.0.3:8983
LESSCLOSE=/usr/bin/lesspipe %s %s
_=/usr/bin/env

I cannot find the one in the error message : SMTPOUT_PORT_25_TCP_ADDR

MLaudren · Answer 14 · Tue Nov 29 2016 21:20:55 GMT+0800 (China Standard Time)

Ok, I've updated the peps.sh to map to SMTPOUT_PORT_465_TCP_ADDR and SMTPOUT_PORT_465_TCP_PORT instead of 25 and now peps server is running.

but still have the domain error in smtpin...

MLaudren · Answer 15 · Tue Nov 29 2016 21:57:04 GMT+0800 (China Standard Time)

I've started again manually the smtpin and it look like good.
but if I go to the webaccess it don't ask for the admin password, but for an account. probably from my previous installation... I just cannot remeber my accounts :/
How can I delete all old data/DB from my previous installation?
Try to remove the volumes, but does not make any changes.

MLaudren · Answer 16 · Wed Nov 30 2016 06:50:53 GMT+0800 (China Standard Time)

Manage to make it works!

Thanks Tiangolo!

Sebastián Ramírez · Answer 17 · Thu Dec 01 2016 02:48:46 GMT+0800 (China Standard Time)

Great @MLaudren ! 🎉

MLaudren · Answer 18 · Thu Dec 01 2016 03:46:29 GMT+0800 (China Standard Time)

Will close the issue after PR #71 is accepted.

MLaudren · Answer 19 · Fri May 04 2018 02:40:49 GMT+0800 (China Standard Time)

Issue solve with PR #69 and PR #71