How do I properly support STARTTLS with aiosmtpd?

Question

I have the following server taken almost directly from the aiosmtpd docs:

import asyncio
import ssl
from aiosmtpd.controller import Controller


class ExampleHandler:
    async def handle_RCPT(self, server, session, envelope, address, rcpt_options):
        if not address.endswith('@example.com'):
            return '550 not relaying to that domain'
        envelope.rcpt_tos.append(address)
        return '250 OK'

    async def handle_DATA(self, server, session, envelope):
        print(f'Message from {envelope.mail_from}')
        print(f'Message for {envelope.rcpt_tos}')
        print(f'Message data:\n{envelope.content.decode("utf8", errors="replace")}')
        print('End of message')
        return '250 Message accepted for delivery'

context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
controller = Controller(ExampleHandler(), port=8026, ssl_context=context)
controller.start()

input('Press enter to stop')
controller.stop()

However, when I start this server and try to send an email to it using swaks:

echo 'Testing' | swaks --to [email protected] --from "[email protected]" --server localhost --port 8026 -tls

It times out after 30s. If I remove the ssl_context=context from the server and -tls from the client then it sends the mail fine.

Additionally, when I try to connect via telnet and just send EHLO whatever then the server actually closes the connection.

What's the correct way to implement an aiosmtpd server that supports tls?

Answer 1

Building upon Wayne's own answer, here's how to create a STARTTLS server with aiosmtpd.

1. Create an SSL context

For testing, use the following command to generate a self-signed certificate for localhost:

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes -subj '/CN=localhost'

Load it into Python using the ssl module:

import ssl
context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context.load_cert_chain('cert.pem', 'key.pem')

2. Pass SSL context to aiosmtpd

Create a subclass of aiosmtpd's Controller that passes this context as the tls_context to SMTP:

from aiosmtpd.smtp import SMTP
from aiosmtpd.controller import Controller

class ControllerTls(Controller):
    def factory(self):
        return SMTP(self.handler, require_starttls=True, tls_context=context)

3. Run it

Instantiate this controller with a handler and start it. Here, I use aiosmtpd's own Debugging handler:

from aiosmtpd.handlers import Debugging
controller = ControllerTls(Debugging(), port=1025)
controller.start()
input('Press enter to stop')
controller.stop()

4. Test it

Either configure a local mail client to send to localhost:1025, or use swaks:

swaks -tls -t test --server localhost:1025

... or use openssl s_client to talk to the server after the initial STARTTLS command has been issued:

openssl s_client -crlf -CAfile cert.pem -connect localhost:1025 -starttls smtp

The full code

The code below additionally tests the server using swaks, and it also shows how to create a TLS-on-connect server (as in Wayne's answer).

import os
import ssl
import subprocess
from aiosmtpd.smtp import SMTP
from aiosmtpd.controller import Controller
from aiosmtpd.handlers import Debugging

# Create cert and key if they don't exist
if not os.path.exists('cert.pem') and not os.path.exists('key.pem'):
    subprocess.call('openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem ' +
                    '-days 365 -nodes -subj "/CN=localhost"', shell=True)

# Load SSL context
context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context.load_cert_chain('cert.pem', 'key.pem')

# Pass SSL context to aiosmtpd
class ControllerStarttls(Controller):
    def factory(self):
        return SMTP(self.handler, require_starttls=True, tls_context=context)

# Start server
controller = ControllerStarttls(Debugging(), port=1025)
controller.start()
# Test using swaks (if available)
subprocess.call('swaks -tls -t test --server localhost:1025', shell=True)
input('Running STARTTLS server. Press enter to stop.\n')
controller.stop()

# Alternatively: Use TLS-on-connect
controller = Controller(Debugging(), port=1025, ssl_context=context)
controller.start()
# Test using swaks (if available)
subprocess.call('swaks -tlsc -t test --server localhost:1025', shell=True)
input('Running TLSC server. Press enter to stop.\n')
controller.stop()

Answer 2

I was close. I figured from the fact that I could connect via telnet, but EHLO hostname would disconnect that the server was trying to require a TLS connection ahead of time.

When I examined swaks --help I found that there was a slightly different option that would probably do what I wanted:

--tlsc, --tls-on-connect
    Initiate a TLS connection immediately on connection.  Following common convention,
    if this option is specified the default port changes from 25 to 465, though this can
    still be overridden with the --port option.

When I tried that, I still got an error:

$ echo 'Testing' | swaks --to [email protected] --from "[email protected]" --server localhost --port 8026 -tlsc
=== Trying localhost:8026...
=== Connected to localhost.
*** TLS startup failed (connect(): error:00000000:lib(0):func(0):reason(0))

Through some of my perusal of the Python ssl documentation, I noticed the load_cert_chain method. It turned out that this was exactly what I needed. Following these instructions I generated a totally insecure self-signed certificate:

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes -subj '/CN=localhost'

Then I added this line:

context.load_cert_chain('cert.pem', 'key.pem')

And now I'm able to send email. For the lazycurious, here's the entire server code:

import asyncio
import ssl
from aiosmtpd.controller import Controller


class ExampleHandler:
    async def handle_RCPT(self, server, session, envelope, address, rcpt_options):
        if not address.endswith('@example.com'):
            return '550 not relaying to that domain'
        envelope.rcpt_tos.append(address)
        return '250 OK'

    async def handle_DATA(self, server, session, envelope):
        print(f'Message from {envelope.mail_from}')
        print(f'Message for {envelope.rcpt_tos}')
        print(f'Message data:\n{envelope.content.decode("utf8", errors="replace")}')
        print('End of message')
        return '250 Message accepted for delivery'

context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context.load_cert_chain('cert.pem', 'key.pem')
controller = Controller(ExampleHandler(), port=8026, ssl_context=context)
controller.start()

input('Press enter to stop')
controller.stop()

Which can be validated with:

echo 'Testing' | swaks --to [email protected] --from "[email protected]" --server localhost --port 8026 -tlsc