Mike Driscoll: PyDev of the Week: Glyph Lefkowitz

Why did you start using Python?

I’m starting to feel like my Python origin story has been told and re-told so many times that people are as bored of me as they are of Wolverine :-). But, here goes:

I was working on a video game, “Twisted Reality”, that was originally written in Java.

Twisted Reality was designed to have authors doing content creation in-engine and online, possibly with players watching them do it. This meant that a lot of behavior had to be dynamic and run-time modifiable, which meant that objects had lots of collection data structures to map item names and verbs to various simulation objects and actions.

I had a summer job writing a calendaring system in Perl, and as I was increasingly having problems with the JVM, I was casting around for other languages I might use. One of the major problems I had with the JVM was that the UIs just behaved incorrectly and were visibly laggy and slow (remember; this was 1999 or so) no matter how much I tweaked them. I started re-writing the client software in C, for Linux, to make native use of the GTK. I believe I originally discovered Python because one of the other developers pointed out that it was possible to write GTK UIs using a much higher-level language, so the first thing I wrote was a PyGTK client for our game server.

One of the things I noticed when writing the client is that many of the high-level dynamic behavior constructs that I had cobbled together by hand in Java were built right in to Python – like the ability to dynamically add an attribute to an object. The others which I used frequently, like hash tables, involved radically less typing; ‘x = {}’ rather than ‘import java.util.Hashtable; Hashtable x = new Hashtable();’.

Since it seemed like it had all the things I needed, soon after rewriting the Twisted Reality client in Python, I re-wrote the server and got about a 20:1 code compression ratio. The rest, of course, is history.

What other programming languages do you know and which is your favorite?

I know a smattering of Bourne shell, Perl, Emacs Lisp, Smalltalk, C, C++, and Java.

My favorite out of that set would probably be Smalltalk. Abstractly, I actually find Smalltalk more beautiful than Python because it manages to build a complete language out of such a small amount of syntax. Practically, of course, no actual Smalltalk implementation has made the necessary compromises with reality to make it as pleasant an experience of writing Python.

What projects are you working on now?

Twisted itself is an enduring and increasingly vibrant community, and I still dedicate a fair amount of time to it. I also work on Mimic which is a server for mocking out API calls; that project is important to me because I’m a big advocate for improved software testing, and one of the most neglected parts of the art of testing is constructing reliable, deterministic fakes which can be used across projects.

Beyond that though, one of the nice things about my research role at Rackspace is that I have the flexibility to experiment with new projects. Sometimes this means some contributing upstream to maintenance of other libraries, and sometimes it means experimental new side-projects. One such side-project is my state-machine library Automat, which allows for easy expression of formal state machines. Another is Horsephrase which is a simple password generator designed to apply a bit more information-theoretical rigor to the process of generating pronounceable, communicable passwords.

Lately, I’ve been working on a series of enhancements to Klein, which is a Twisted based web micro-framework, to make it more competitive with other frameworks, and make it more featureful and familiar to users of such frameworks.

Which Python libraries are your favorite (core or 3rd party)?

As recently, famously covered on my blog, I think ‘attrs’ is a fantastic library that everyone should be using. It’s still got some evolving to do but I really hope it becomes part of the standard library at some point.

Where do you see Python going as a programming language?

I think we’re at a crossroads.

One of the things that makes Python so appealing is its generality. It’s the second-best language for everything; if you want the absolute best language for the web, for example, in terms of availability of libraries and developers and time to market, Ruby – due to Rails – might be a better bet. If you want the best language for enterprise computing or distributed systems, Java, or at least, something in the JVM ecosystem, might be better than Python. If you want to do statistics, R or MATLAB might have more functionality available.

But, if you want to write some numerical analysis that is hooked up to a web app which needs to do reporting from an enterprise database, Python will beat any of these languages hands down.

So, as a community, I think we have two choices before us. Either we are going to keep pushing boundaries, keep making Python more general – make it run in the browser, for example, make it run better on mobile devices – or we are going to be complacent about the places where it’s already doing well.

Right now, it’s possible to run Python almost anywhere, for almost any application. But that doesn’t mean it’s practical, or accepted. If you go to do front-end stuff, there’s a strong expectation you’ll be using a JavaScript framework like React. That’s why I think issues like this one are so important. Back in the old days, Python was competing against really horrible alternatives, which it could beat handily: giant piles of closed-source C++, for example. Now, the competition is much more intense: ECMAScript 7 with React in the browser, for example, and Clojure with advanced features like JMX on the back end.

Right now I’m concerned about front-end stuff, because that seems to be the area where Python is the weakest and also the area where there is the most potential for another breakout success. (And frankly because I want to write cool front-end stuff and I don’t have a huge amount of time to invest.)

I think there needs to be a big focus on developer experience and out-of-the-box performance. But there are other areas where we have already made progress, yet the community isn’t picking up on it. For example, many blogs still come out saying things like “python is slow”, despite the fact that PyPy’s performance is very competitive with other language runtimes. I think more people need to be checking out PyPy; for most server-side applications, it makes no sense to use the relatively slow, memory-hungry CPython by default. There are certain C extensions that still require CPython, of course, and so there’s nothing wrong with using it, if it works well for you. But CPython is a bad default when programmers are still getting it as the default, benchmarking it, and then declaring “python is slow, we’re going to rewrite this whole thing in Go” before even finding out about PyPy.

What is your take on the current market for Python programmers?

I don’t have a lot of expertise here, but I can read a graph. These graphs all say the same thing:

• http://www.tiobe.com/tiobe-index/
• http://pypl.github.io/PYPL.html
• http://redmonk.com/sogrady/2016/07/20/language-rankings-6-16/

No matter how you count, Python is among the top 10 programming languages, and it continues to be an exciting and dynamic ecosystem to participate in. Frankly, I was concerned for quite a while about the 2/3 schism and I thought it might seriously hurt Python in the long term. But despite the churn that that ongoing migration process has caused, Python continues to grow, and the library ecosystem continues to evolve with the times, even while continuing to support Python 2.

Is there anything else you’d like to say?

One of my favorite parts of the Python community is its commitment to diversity. As a result of this commitment, it’s one of the friendliest, most welcoming open source communities out there. We often – rightly – pat ourselves on the back for progress in this area.

However, one thing I think it’s important to speak out about – and since you’ve given me a space here explicitly to speak out on something I think is important, I should do so – is that we absolutely can’t stop here. We still need to be welcoming.

An easy way to do this is to work on friendliness and openness to beginners and unfamiliar people in your open source communities. Thank every contributor for their patches – even if the patches are terrible, they still dedicated time to learning enough about your project to submit one, and that investment of time is a vote of confidence in your work! This goes the other way, too – sometimes the users have more status and power than the maintainers. Be sure to thank maintainers for their efforts, especially if they’re volunteers.

But just doing the easy stuff isn’t good enough. Leaders in open source communities need to participate in outreach programs. Managers in software companies need to actively work to remove bias and unfairness (which are not necessarily the same thing!) from their hiring processes.

In summary: good job so far, Python community, but we need to keep up the good work!