UDP flow control with Python Twisted

Question

I have a class that inherits from twisted.internet.protocol.DatagramProtocol class. In my startProtocol() implementation I call startWriting(), so that socket gets notified each time I can write to it without blocking. Two questions:

1. Which method twisted will call once socket will become writable?
2. How should I call startWriting() method, if it needs to be called on particular time-intervals to limit the outgoing UDP bandwidth to particular Datagrams/Second amount?

Answer 1

Oops. I think I may have answered your question in another thread and lead you to believe that UDP flow control support in Twisted is a little more robust than it actually is. Nevertheless, you can get what you need to do done...

1. Which method will Twisted call once socket will become writable?

Unfortunately, UDP protocols in Twisted do not get monitored for writability, on the premise that UDP can always fail and so it should never raise EWOULDBLOCK. (Except actually, it does sometimes, and this is a bug in Twisted which I just re-discovered while answering this question. This only happens if Twisted is sending UDP at faster than local wire speed, which requires a very fast application and a very slow network.)

As a workaround, your application can simply catch EWOULDBLOCK. For any other protocol, this type of workaround might constitute a serious problem, but in the case of UDP you already have to be prepared to lose any outgoing packets, so you need an in-band control flow mechanism anyway.

Helping us get that bug through the review process is always an option too.

If you want to get really fancy, you can write your own alternative to udp.Port (by implementing IFileDescriptor yourself) rather than writing a UDP protocol, and override doRead and doWrite (which are called when the underlying socket is readable and writable respectively). This will give you perfect write-level flow control, but probably isn't necessary since, again, UDP will just drop your packets sometimes, and on networks without the ability to properly process "ICMP source quench" messages (which dumb firewalls configured to block ICMP will just block), dropped packets are your only source of flow control information. I'm not saying that you shouldn't fix this bug in Twisted for real, but this fact of life in the UDP world is the likely reason that nobody has bothered to do so yet.

2. How should I call startWriting() method, if it needs to be called on particular time-intervals to limit the outgoing UDP bandwidth to particular Datagrams/Second amount?

Due to the limitations described in part 1 of this answer, UDP transports do not have a useful startWriting method.

However, startWriting/stopWriting isn't really the right way to time-limit your outgoing UDP bandwidth anyway.

Simply call self.transport.write(...) at the appropriate times, after scheduling said calls with via an appropriate scheduling mechanism. LoopingCall, for example, was designed to invoke UDP sends for RTP media streaming at the appropriate interval for transmitting a sound sample. But you can also just calculate your own delay and use callLater directly. In any case, you likely will need to keep around some kind of queue mechanism in the event that you need to do any re-transmissions of outgoing data queued for transport over UDP, so just pop

If you need to do inbound flow-control, UDP transports still support that quite nicely, with stopReading and startReading.

Hope this answer was helpful, and sorry if I previously mislead you about Twisted's capabilities in this area!