MailKit fails on Unicode password

[Axel D.]

Describe the bug

I created an e-mail mailbox using a password containing Unicode characters.

While I can easily sign-in to my e-mail mailbox using the browser UI, trying to access it using MailKit fails with "535 Authentication failed."

Platform (please complete the following information):

• OS: Windows 10
• .NET Runtime: .NET 7
• .NET Framework: .Net Core
• MailKit Version: 4.2.0

Exception

Message: 
Test method WebApiTests.ContactFormHandlerTests.SendEMailTests.SuccessfullySendEMailTest threw exception: 
MailKit.Security.AuthenticationException: 535: Authentication failed. Restarting authentication process. ---> MailKit.Net.Smtp.SmtpCommandException: Authentication failed. Restarting authentication process.

  Stapelüberwachung: 
SmtpClient.AuthenticateAsync(Encoding encoding, ICredentials credentials, CancellationToken cancellationToken)
MailKitSmtpSenderService.SendMailMessage(MailMessage message) Zeile 57
ContactFormHandler.SendMessage(IPAddress ip, String token, String result, Byte minAuthoringTimeSeconds, MailMessage message, IMailSender[] sendProviders, Boolean isTestEnvironment) Zeile 415
SendEMailTests.SuccessfullySendEMailTest() Zeile 231

Expected behavior

I should be able to connect to my e-mail server and send an e-mail message.

Code Snippets

private MailKitSmtpSenderConfig _config = new MailKitSmtpSenderConfig()
{
  "HostName": "…",
  "Port": "587",
  "UserName": "[e-mail]",
  "Password": "^1a°!B"
};

public async Task SendMailMessage(System.Net.Mail.MailMessage message)
{
  using MemoryStream stream = new MemoryStream(10_240);
  using StreamReader reader = new StreamReader(stream);
  using ProtocolLogger logger = new ProtocolLogger(stream);
  using SmtpClient smtp = new SmtpClient(logger);

  try
  {
    await smtp.ConnectAsync(
      _config.HostName,
      _config.Port,
      SecureSocketOptions.StartTlsWhenAvailable
      );
    await smtp.AuthenticateAsync(_config.UserName, _config.Password);
    await smtp.SendAsync(new MimeMessage(message));
    await smtp.DisconnectAsync(true);
  }
  catch
  {
    stream.Position = 0L;
    Debug.Print(reader.ReadToEnd());
  }
}

Protocol Logs

Connected to smtp://…:587/?starttls=when-available
S: 220 …
C: EHLO DevTest
S: 250-…
S: 250-SIZE 52428800
S: 250-STARTTLS
S: 250-AUTH LOGIN
S: 250 HELP
C: STARTTLS
S: 220 Ready to start TLS
C: EHLO DevTest
S: 250-…
S: 250-SIZE 52428800
S: 250-AUTH LOGIN
S: 250 HELP
C: AUTH LOGIN
S: 334 VXNlcn5gbWU6
C: ********
S: 334 UGFzj3dvclQ6
C: ********
S: 535 Authentication failed. Restarting authentication process.

[Jeffrey Stedfast]

Most likely the server expects the username and password to be in a non-UTF-8 encoding (like maybe iso-8859-1).

Notice how your SMTP server does not support the SMTPUTF8 extension, which usually indicates that the server doesn't support UTF-8 anything, including authentication credentials.

Try calling await smtp.AuthenticateAsync(System.Text.Encoding.GetEncoding("iso-8859-1"), _config.UserName, _config.Password);

If that doesn't work, try other encodings that might match the server's locale encoding until you find one that works.

[Jeffrey Stedfast]

Another common one, especially in Western Europe, is iso-8859-15.

[Axel D.]

@jstedfast:

Thank you for your support!

Wouldn't it be feasible if MailKit automatically tried a number of common encodings if the password contained non-ASCII characters and the SMTP server doesn't come with a SMTPUTF8 extension? Or, alternatively, throw an exception?

[Jeffrey Stedfast]

There's like 100 different encodings to try...