This is a follow-on from my
previous post
on Python 3.5's new async/await syntax. Rather than the simple background
timers used in the original post, this one will look at the impact native
coroutine support has on the TCP echo client and server examples from the
asyncio documentation.
First, we'll recreate the run_in_foreground helper defined in the previous
post. This helper function makes it easier to work with coroutines from
otherwise synchronous code (like the interactive prompt):
defrun_in_foreground(task,*,loop=None):"""Runs event loop in current thread until the given task completes Returns the result of the task. For more complex conditions, combine with asyncio.wait() To include a timeout, combine with asyncio.wait_for()"""ifloopisNone:loop=asyncio.get_event_loop()returnloop.run_until_complete(asyncio.ensure_future(task))
Next we'll define the coroutine for our TCP echo server implementation, which simply waits to receive up to 100 bytes on each new client connection, and then sends that data back to the client:
asyncdefhandle_tcp_echo(reader,writer):data=awaitreader.read(100)message=data.decode()addr=writer.get_extra_info('peername')print("-> Server received %r from %r"%(message,addr))print("<- Server sending: %r"%message)writer.write(data)awaitwriter.drain()print("-- Terminating connection on server")writer.close()
And then the client coroutine we'll use to send a message and wait for a response:
asyncdeftcp_echo_client(message,port,loop=None):reader,writer=awaitasyncio.open_connection('127.0.0.1',port,loop=loop)print('-> Client sending: %r'%message)writer.write(message.encode())data=(awaitreader.read(100)).decode()print('<- Client received: %r'%data)print('-- Terminating connection on client')writer.close()returndata
We then use our run_in_foreground helper to interact with these coroutines
from the interactive prompt. First, we start the echo server:
>>> make_server=asyncio.start_server(handle_echo,'127.0.0.1')>>> server=run_in_foreground(make_server)
Conveniently, since this is a coroutine running in the current thread, rather than in a different thread, we can retrieve the details of the listening socket immediately, including the automatically assigned port number:
>>> server.sockets[0]<socket.socket fd=6, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 40796)>>>> port=server.sockets[0].getsockname()[1]
Since we haven't needed to hardcode the port number, if we want to define a second server, we can easily do that as well:
>>> make_server2=asyncio.start_server(handle_tcp_echo,'127.0.0.1')>>> server2=run_in_foreground(make_server2)>>> server2.sockets[0]<socket.socket fd=7, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 41200)>>>> port2=server2.sockets[0].getsockname()[1]
Now, both of these servers are configured to run directly in the main thread's event loop, so trying to talk to them using a synchronous client wouldn't work. The client would block the main thread, and the servers wouldn't be able to process incoming connections. That's where our asynchronous client coroutine comes in: if we use that to send messages to the server, then it doesn't block the main thread either, and both the client and server coroutines can process incoming events of interest. That gives the following results:
>>> print(run_in_foreground(tcp_echo_client('Hello World!',port)))-> Client sending: 'Hello World!'-> Server received 'Hello World!' from ('127.0.0.1', 44386)<- Server sending: 'Hello World!'-- Terminating connection on server<- Client received: 'Hello World!'-- Terminating connection on clientHello World!
Note something important here: you will get exactly that sequence of output messages, as this is all running in the interpreter's main thread, in a deterministic order. If the servers were running in their own threads, we wouldn't have that property (and reliably getting access to the port numbers the server components chose would also have been far more difficult).
And to demonstrate both servers are up and running:
>>> print(run_in_foreground(tcp_echo_client('Hello World!',port2)))-> Client sending: 'Hello World!'-> Server received 'Hello World!' from ('127.0.0.1', 44419)<- Server sending: 'Hello World!'-- Terminating connection on server<- Client received: 'Hello World!'-- Terminating connection on clientHello World!
That then raises an interesting question: how would we send messages to the
two servers in parallel, while still only using a single thread to manage the
client and server coroutines? For that, we'll need our other helper function
from the previous post, run_in_background:
defrun_in_background(target,*,loop=None,executor=None):"""Schedules target as a background task Returns the scheduled task. If target is a future or coroutine, equivalent to asyncio.ensure_future If target is a callable, it is scheduled in the given (or default) executor"""ifloopisNone:loop=asyncio.get_event_loop()try:returnasyncio.ensure_future(target,loop=loop)exceptTypeError:passifcallable(target):returnloop.run_in_executor(executor,target)raiseTypeError("background task must be future, coroutine or ""callable, not {!r}".format(type(target)))
First, we set up the two client operations we want to run in parallel:
>>> echo1=run_in_background(tcp_echo_client('Hello World!',port))>>> echo2=run_in_background(tcp_echo_client('Hello World!',port2))
Then we use the asyncio.wait function in combination with run_in_foreground
to run the event loop until both operations are complete:
>>> run_in_foreground(asyncio.wait([echo1,echo2]))-> Client sending: 'Hello World!'-> Client sending: 'Hello World!'-> Server received 'Hello World!' from ('127.0.0.1', 44461)<- Server sending: 'Hello World!'-- Terminating connection on server-> Server received 'Hello World!' from ('127.0.0.1', 44462)<- Server sending: 'Hello World!'-- Terminating connection on server<- Client received: 'Hello World!'-- Terminating connection on client<- Client received: 'Hello World!'-- Terminating connection on client
And finally, we retrieve our results using the result method of the future
objects returned by run_in_background:
>>> echo1.result()'Hello World!'>>> echo2.result()'Hello World!'
We can set up as many background tasks as we like, and then use asyncio.wait
as the foreground task to wait for them all to complete.