Giters

sisimai / p5-sisimai

Mail Analyzing Interface for email bounce: A Perl module to parse RFC5322 bounce mails and generating structured data as JSON from parsed results. Formerly known as bounceHammer 4: an error mail analyzer.

Home Page:https://libsisimai.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Incorrect parsing of email - not catching a softbounce

meir-w opened this issue · comments

commented

I am getting an incorrect bounce report.

"deliverystatus": "5.1.0",
"recipient": "XXXXX@YYY.ZZZ",
"subject": "The best walking workout for older people",
"listid": "",
"feedbacktype": "",
"diagnostictype": "SMTP",
"timezoneoffset": "+0000",
"origin": "<STDIN>",
"addresser": "wellbeing@XXX.YYY.ZZZ",
"destination": "charter.net",
"diagnosticcode": "550 5.1.0 <01000172586829c9-830041fa-7772-4fe0-9bd7-53b6f56bdb55-000000@amazonses.com> sender rejected. Please see https://www.spectrum.net/support/internet/understanding-email-error-codes for more information. AUP#In-1310",
"smtpcommand": "",
"smtpagent": "AmazonSES",
"replycode": "550",
"messageid": "01000172586829c9-830041fa-7772-4fe0-9bd7-53b6f56bdb55-000000@email.amazonses.com",
"timestamp": 1590606721,
"senderdomain": "XXX.YYY.ZZZ",
"alias": "",
"token": "7d68c34bd18b382e5addf48c144e4b7b913cc8e8",
"catch": null,
"reason": "userunknown",
"lhost": "d217-29.smtp-out.amazonses.com",
"softbounce": 0,
"action": "failed",
"rhost": "d217-29.smtp-out.amazonses.com"

},

The diagnostic code indicates it is a sender rejected message. This should be considered a soft bounce. Yet it is being marked a hard bounce, with reason userunknown. But that is not the reason given, nor the meaning of the SMTP code.

commented

Thanks for the report. We have some questions about the parsed result:

  1. Is the recipient address a really existent email address?
  2. Can you open the URL: https://www.spectrum.net/support/internet/understanding-email-error-codes? We can open neither the URL nor https://www.spectrum.net/. Both URLs returns 403 Forbidden.
  3. Would you post the entire email message including all the headers into this issue? Because the entire message is needed to inspect details of a bounce.

We think the correct bounce reason may be rejected (sender address rejected), However, we cannot see details of error code: AUP#In-1310 at spectrum.net.

Best regards,

commented

spectrum.net always return 403 (accessed from Japan)

% wget https://www.spectrum.net/support/internet/understanding-email-error-codes
--2020-06-01 19:13:05--  https://www.spectrum.net/support/internet/understanding-email-error-codes
Resolving www.spectrum.net (www.spectrum.net)... 52.55.80.46, 34.197.32.148, 52.20.194.46
Connecting to www.spectrum.net (www.spectrum.net)|52.55.80.46|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2020-06-01 19:13:06 ERROR 403: Forbidden.
commented

And yes it is for a real user, who normally receives email.
I can post the full body (or a truncated version that still returns the same result) if that would help

commented

Thanks for the information about AUP#In-1310 :-)

We found that the page says: AUP#1310: Cloudmark related error indicating a 5 minute rate limit block. If it is so, the correct bounce reason may be one of the followings:

We'll try to implement these error codes in Sisimai::Reason::TooManyConn class or Sisimai::Reason::Blocked class within a few days. We may beg you to send the entire message to us when the fixed code does not work properly.

Best regards,

commented

Here's a cleaned version of the email - I left all the headers and basic structure in place
cleaned.txt

We have many such examples for many users.

commented

We have started to implement code to resolve this issue. And then, we found SMTP error codes on Spectrum at https://www.spectrumbusiness.net/support/internet/understanding-email-error-codes

By the way, error code beginning with AUP# seem to use not only in Spectrum but also in cox.com: https://www.cox.com/residential/support/email-error-codes.html

We have a question about cleaned.txt you give.
Does any Received: header of the original message was removed before post to this issue or did not exist when the bounce mail has been received?

Best regards,

commented

I gave you the all the headers - just changed the domains, and content of the original message body.
A bit confused by your question though. There is a Received header in the file, for the bounce message.
Received: from a14-88.smtp-out.amazonses.com (a14-88.smtp-out.amazonses.com [54.240.14.88])

If you mean from the original message, there was no such header present.

commented

I’m sorry for lack of explanation. The Received header is a header after the Content-Type: message/rfc822 line.

Thanks,

commented
commented

We've implemented Sisimai::Rhost::Spectrum to parse the bounce from Spectrum correctly. The code is available at the branch: issue-395-error-code-aup https://github.com/sisimai/p5-sisimai/tree/issue-395-error-code-aup (Not merged yet).

Would you mind if we add the file cleaned.txt into set-of-emails/ directory of the repository? Because we want to implement a test code for Sisimai::Rhost::Spectrum.

The following is the result of cleaned.txt parsed by the new code.

[
  {
    "softbounce": 1,
    "feedbacktype": "",
    "action": "failed",
    "catch": {
      "x-mailer": "PHPMailer 5.2.7 (https://github.com/PHPMailer/PHPMailer/)",
      "queue-id": "",
      "sender": ""
    },
    "destination": "charter.net",
    "lhost": "d217-29.smtp-out.amazonses.com",
    "timezoneoffset": "+0000",
    "diagnostictype": "SMTP",
    "listid": "",
    "alias": "",
    "deliverystatus": "5.1.0",
    "rhost": "d217-29.smtp-out.amazonses.com",
    "messageid": "010001725d8ad4fc-9fe8fb1f-0292-430b-a634-1eba65166dbb-000000@email.amazonses.com",
    "token": "a7af3b9d80f602011691fb46ba73b298af37a284",
    "replycode": "550",
    "smtpcommand": "",
    "timestamp": 1590739679,
    "reason": "toomanyconn",
    "smtpagent": "AmazonSES",
    "recipient": "theusername@charter.net",
    "origin": "issue-395.eml",
    "addresser": "wellbeing@mrk1.example.com",
    "senderdomain": "mrk1.example.com",
    "diagnosticcode": "550 5.1.0 <010001725d8ad4fc-9fe8fb1f-0292-430b-a634-1eba65166dbb-000000@amazonses.com> sender rejected. Please see https://www.spectrum.net/support/internet/understanding-email-error-codes for more information. AUP#In-1310",
    "subject": "11 things that can happen when you drink apple cider vinegar every day"
  }
]
commented
commented

That would be great. I stripped anything related to our company and the
customer, so you could use it freely.

@meir-w Thank you so much :-)

commented

We've merged the branch and tagged with v4.25.7p3 just now. v4.25.7p3 ( including the new code) is available at https://github.com/sisimai/p5-sisimai/releases/tag/v4.25.7p3

Thank you again :-)