Carl Kadie, Ph.D., is a research developer in Microsoft Research/TnR working on Genomics.
Lambda expressions provide a way to pass functionality into a function. Sadly, Python puts two annoying restrictions on lambda expressions. First, lambdas can only contain an expression, not statements. Second, lambdas can’t be serialized to disk. This blog shows how we can work around these restrictions and unleash the full power of lambdas.
So what are lambda’s good for? Suppose, you have a list of words from a string.
"This is a test string from Carl".split()You can sort the words with sorted().
sorted("This is a test string from Carl".split())Notice, however, that all the capitalized words, sort before all the lower-case words. This can be fixed by passing a lambda expression as the key argument to the sorted() function.
sorted("This is a test string from Carl".split(),key=lambdaword:word.lower())# `key=str.lower` also works.Lambda can be more complicated. Suppose we want to sort the words based on their (lower-case) back-to-front letters? As a reminder, here is a Python way to reverse the lower-case letters of a word:
str.lower("Hello")[::-1]And here is how to pass this functionality to sorted() using a lambda:
sorted("This is a test string from Carl".split(),key=lambdaword:word.lower()[::-1])But what if you want even more complex functionality? For example, functionality that requires if statements and multiple lines with unique scoping? Sadly, Python restricts lambdas to expressions only. But there is a workaround!
Define a function that
- defines an inner function and …
- returns that inner function.
Note that the inner function can refer to variables in the outer function, giving you that private scoping.
In this example lower_sorted() is the outer function. It has an argument called back_to_front. Inside lower_sorted, we define and return an inner function called inner_lower_sorted(). That inner function has multiple lines including an if statement that references back_to_front.
deflower_sorted(back_to_front=False):definner_lower_sorted(word):result=word.lower()ifback_to_front:#The inner function can refer to outside variablesresult=result[::-1]returnresultreturninner_lower_sortedprint(sorted("This is a test string from Carl".split(),key=lower_sorted()))print(sorted("This is a test string from Carl".split(),key=lower_sorted(back_to_front=True)))You may find lambdas and these inner functions handy enough that you’d like to serialize one to disk for use later. Sadly, if you try to seralize with pickle module, you’ll get an error message like “TypeError: can’t pickle function objects”.
A nice workaround is to use the dill project in place of pickle. The dill project is a third-party package that is now included in the standard Anaconda distribution. Here is an example:
!pip install dill
importdillaspicklewithopen("temp.p",mode="wb")asf:pickle.dump(lower_sorted(back_to_front=True),f)withopen("temp.p",mode="rb")asf:some_functionality=pickle.load(f)sorted("This is a test string from Carl".split(),key=some_functionality)Serialization of lambdas and these inner functions opens exciting possibilities. For example, we use it in one of our libraries to run work in different processes and even on different machines in a cluster.
We’ve seen that lambdas are a handy way to pass functionality into a function. Python’s implementation of lambdas has two restrictions, but each restriction has a workaround.
- Multiple lines not allowed.
- Workaround: Define a function that defines and returns an inner function. The inner function can use variables outside itself.
- Can’t pickle lambdas or inner functions.
- Workaround: Replace
picklewithdill.
- Workaround: Replace
Python offers features such as list comprehensions that makes lambdas less used that in other languages. When you do need lambdas, however, they will now be unleashed.