Mike Driscoll: Restarting a Twisted Reactor

I recently started using twisted a couple of weeks ago. For those who don’t know, twisted is “event-driven networking engine written in Python”. The learning curve is pretty steep if you’ve never done asynchronous programming before. During the project I was working on, I ran into a condition where I thought I needed to restart the twisted reactor. According to everything I found online, restarting the reactor is not supported. But I can be stubborn so I tried to find a way anyway.

Restarting a Twisted Reactor

Let’s start by creating a pretty standard twisted server. We’ll subclass LineReceiver which basically makes a TCP server that accepts full lines of text, although it can also do raw data too. Let’s take a look at the code:

from twisted.internetimport reactor
from twisted.protocols.basicimport LineReceiver
from twisted.internet.protocolimport Factory
 
PORT = 9000 
class LineServer(LineReceiver):
 
    def connectionMade(self):
        """
        Overridden event handler that is called when a connection to the 
        server was made
        """print"server received a connection!" 
    def connectionLost(self, reason):
        """
        Overridden event handler that is called when the connection 
        between the server and the client is lost
        @param reason: Reason for loss of connection
        """print"Connection lost"print reason
 
    def lineReceived(self, data):
        """
        Overridden event handler for when a line of data is 
        received from client
        @param data: The data received from the client
        """print'in lineReceived'print'data => ' + data
 
 
class ServerFactory(Factory):
    protocol = LineServer
 
 
if __name__ == '__main__':
    factory = ServerFactory()
    reactor.listenTCP(PORT, factory)
    reactor.run()

All of the camel-cased methods are overridden twisted methods. I just stubbed them out and have them print to stdout whenever they get called. Now let’s make a client that has a reactor that we restart a few times:

importtimeimport twisted.internet 
from twisted.internetimport reactor, protocol
from twisted.protocols.basicimport LineOnlyReceiver
 
HOST = 'localhost'
PORT = 9000 
class Client:
    """
    Client class wrapper
    """def__init__(self, new_user):
        self.new_user = new_user
 
        self.factory = MyClientFactory() 
    def connect(self, server_address=HOST):
        """
        Connect to the server
        @param server_address: The server address
        """
        reactor.connectTCP(server_address, PORT, self.factory,
            timeout=30) 
class MyProtocol(LineOnlyReceiver):
 
    def connectionMade(self):
        """
        Overridden event handler that is fired when a connection
        is made to the server
        """print"client connected!"self.run_long_running_process() 
    def lineReceived(self, data):
        """
        Gets the data from the server
        @param data: The data received from the server
        """print"Received data: " + data
 
    def connectionLost(self, reason):
        """
        Connection lost event handler
        @param reason: The reason the client lost connection 
            with the server
        """print"Connection lost" 
    def run_long_running_process(self):
        """
        Run the process
        """print'running process'time.sleep(5)print"process finished!"self.transport.write('finished' + '\r\n')
        reactor.callLater(5, reactor.stop) 
class MyClientFactory(protocol.ClientFactory):
    protocol = MyProtocol
 
 
if __name__ == '__main__':
    # run reactor multiple times
    tries = 3while tries:
        client = Client(new_user=True)
        client.connect('localhost')try:
            reactor.run()
            tries -= 1print"tries " + str(tries)exceptException, e:
            print e
            importsysdelsys.modules['twisted.internet.reactor']from twisted.internetimport reactor
            from twisted.internetimport default
            default.install()

Here we create a simple client that accepts only lines of text too (LineOnlyReceiver). The magic for restarting the reactor lies in the while loop at the end of the code. I actually found the code in the exception handler inside of twisted’s reactor.py file which gave me the idea. Basically what we’re doing is importing Python’s sys module. We then delete the reactor from sys.modules which allows us to re-port it and reinstall the default reactor. If you run the server in one terminal and the client in another, you can see the client reconnect three times.

Wrapping Up

As I mentioned at the beginning, I’m still pretty new to twisted. What you should probably do instead of restarting the reactor is run it in another thread. Or you might use one of its delayed calls or deferred threads to get around the “need” to restart the reactor. Frankly, the method in this article doesn’t even work in some conditions. I was actually trying to restart the reactor once inside a function that was decorated with contextlib’scontextmanager decorator and that somehow prevented this code to work correctly. Regardless, I thought it was an interesting way to reload a module.