python - When are WebSocketHandler and TornadoWebSocketClient completely deleted?

Question

I'm working on an application that must support client-server connections. In order to do that, I'm using the module of tornado that allows me to create WebSockets. I intend to be always in operation, at least the server-side. So I am very worried about the performance and memory usage of each of the objects created on these connections. I have started to do tests to detect when actually these objects are eliminated by the library. Take the example code and i overwrote the method __del__()

server.py

#! /usr/bin/env python
import tornado.httpserver
import tornado.websocket
import tornado.ioloop
import tornado.web
import gc, sys
import resource

class WSHandler(tornado.websocket.WebSocketHandler):
    def open(self):
        print 'new connection'
        self.write_message("h")

    def check_origin(self, origin):
        return True

    def on_message(self, message):
        print "Message: " + message

    def on_close(self):
        print 'Closed'
        print 'GC count: ' + str(len(gc.get_referrers(self)))

    def __del__(self):
        print "DELETED"

application = tornado.web.Application([
    (r'/s', WSHandler),
])


if __name__ == "__main__":
    http_server = tornado.httpserver.HTTPServer(application)
    http_server.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

client.py

#! /usr/bin/env python
from ws4py.client.tornadoclient import TornadoWebSocketClient
from tornado import ioloop

class MainClient(TornadoWebSocketClient):
     def opened(self):
        print "Connected"

     def received_message(self, message):
        print "Message"

        #I close the connection
        self.close()

     def closed(self, code, reason=None):
        print "Closed"
        ioloop.IOLoop.instance().stop()

     def __del__(self):
        print "DELETED"

if __name__ == "__main__":
    ws = MainClient('ws://localhost:8888/s', protocols=['http-only', 'chat'])
    ws.connect()

    ioloop.IOLoop.instance().start()

When the client receives a message, it closes the connection. I hoped that both objects were eliminated, because the connection was closed, and so call the __del__() method, but that didn't happened.

server output:

new connection
Closed
GC count: 6

client output:

Connected
Message
Closed

As you can see it didn't print the DELETED sentence that i was expecting from the __del__() method.

--edited--

Also I've added the line that prints the number of references which has the GC of that object at the time of closing the connection. This proves that there are indeed references cycles.

-----

Obviously the classes that I will use will be more complex than those, but help me to understand the behavior of both objects, which is what I really seek to know: when are they deleted? It frees the memory when remove them? or otherwise becoming somehow? or ¿how delete the object explicitly?

I read the tornado.websocket.WebSocketHandler documentation, it explain me when the object is "closed", but i don't know when the memory is released.

Answer 1

The WebSocket code currently contains some reference cycles, which means that objects are not cleaned up until the next full GC. Even worse, __del__ methods can actually prevent the deletion of an object (in python 3.3 and older: https://docs.python.org/3.3/library/gc.html#gc.garbage), so it's difficult to tell when things are actually getting deleted. Instead, you'll need to just load test your system and see if its memory footprint increases over time.

(Patches to break up the reference cycles after a connection is closed would be welcome)

Answer 2

For me setting the ping interval/timeout solved the issue:

application = tornado.web.Application([(r'/ws', WSHandler), ], websocket_ping_interval=60, websocket_ping_timeout=180)