I've been spending a lot of time practicing on LeetCode recently, so I thought I'd share some of my favorite intermediate-level Python tricks with an emphasis on newer features of Python you may not have started using yet.
I'll start with basic tips and then move to more advanced.
Get help()
Python's documentation is pretty great, and some of these examples are taken from there.
For instance, if you just google "heapq", you'll see the official docs for heapq, which are often enough.
However, it's also helpful to sometimes just quickly get help() in the shell. Here, I can't remember how to insert works:
>>> help([])
>>> dir([])
>>> help([].insert)
enumerate()
If you need to loop over a list, you can use enumerate() to get both the item as well as the index. As a pneumonic, I like to think for (i, x) in enumerate(...):
for (i, x) in enumerate(some_list):
...
items()
Similarly, you can get both the key and the value at the same time when looping over a dict using items():
for (k, v) in some_dict.items():
...
[] vs. get()
Remember, when you use [] with a dict, if the value doesn't exist, you'll get a KeyError. Rather than see if an item is in the dict and then look up its value, you can use get():
val = some_dict.get(key) # It defaults to None.
if val is None:
...
Similarly, .setdefault() is sometimes helpful.
Some people prefer to just use [] and handle the KeyError since exceptions aren't as expensive in Python as they are in other languages.
range() is smarter than you think
for item in range(items):
...
for index in range(len(items)):
...
# Count by 2s.
for i in range(0, 100, 2):
...
# Count backwards from 100 to 0 inclusive.
for i in range(100, -1, -1):
...
# Surprisingly, ranges can even be reversed cheaply.
r = range(100)
r = r[::-1] # range(99, -1, -1)
print(f'') debugging
Have you switched to Python's new format strings yet? They're nicer than % and .format():
print(f'Got item: {item}')
Use a list as a stack
The cost of using a list as a stack is (amortized) O(1):
elements = []
elements.append(element) # Not push
element = elements.pop()
Note that inserting something at the beginning of the list or in the middle is more expensive because you have to shift everything to the right.
sort() vs. sorted()
# sort() sorts a list in place.
my_list.sort()
# Whereas sorted() returns a sorted *copy* of an iterable:
my_sorted_list = sorted(some_iterable)
set and frozenset
Sets are just so useful in so many problems. Just in case you didn't know some of these tricks:
# There is now syntax for creating sets.
s = {'Von'}
# There are set "comprehensions" which are like list comprehensions, but for sets.
s2 = {f'{name} the III' for name in s}
{'Von the III'}
# If you need an immutable set, for instance, to use as a dict key, use frozenset.
frozenset((1, 2, 3))
deque
If you find yourself needing a queue, or a list that you can push and pop from either side, use a deque:
>>> from collections import deque
>>>
>>> d = deque()
>>> d.append(3)
>>> d.append(4)
>>> d.appendleft(2)
>>> d.appendleft(1)
>>> d
deque([1, 2, 3, 4])
>>> d.popleft()
1
>>> d.pop()
4
Using a stack instead of recursion
Instead of using recursion (which has a depth of about 1000 frames), you can use a while loop and manually manage a stack yourself:
work = []
while work:
work_item = work.pop()
piece1, piece2 = process(work_item)
work.push(piece1)
work.push(piece2)
Pre-initialize your list
If you know how long your list is going to be ahead of time, you can avoid needing to resize it multiple times by just pre-initializing it:
dp = [None] * len(items)
collections.Counter()
How many times have you used a dict to count up something. It's built in in Python:
>>> from collections import Counter
>>> c = Counter('abcabcabcaaa')
>>> c
Counter({'a': 6, 'b': 3, 'c': 3})
defaultdict
Similarly, there's defaultdict:
>>> from collections import defaultdict
>>> d = defaultdict(list)
>>> d['girls'].append('Jocylenn')
>>> d['boys'].append('Greggory')
>>> d
defaultdict(<class 'list'>, {'girls': ['Jocylenn'], 'boys': ['Greggory']})
Notice that I didn't need to set d['girls'] to an empty list before I started appending to it.
heapq
I had heard of heaps in school, but I didn't really know what they were. Well, it turns out they're pretty helpful for several of the problems, and Python has a list-based heap implementation built in.
For of all, if you don't know what a heap is, I recommend this video and this video. They'll explain what a heap is and how to implement one using a list.
The heapq module is a built-in module for managing a heap. It builds on top of an existing list:
import heapq
some_list = ...
heapq.heapify(some_list)
# The head of the heap is some_list[0].
# The len of the heap is still len(some_list).
heapq.heappush(some_list, item)
head_item = heapq.heappop(some_list)
The heapq module also has nlargest and nsmallest built in so you don't have to implement those things yourself.
Keep in mind that heapq is a minheap. Let's say that what you really want is a maxheap, and you're not working with ints, you're working with objects. Here's how to tweak your data to get it to fit heapq's way of thinking:
heap = []
heapq.heappush(heap, (-obj.value, obj))
(ignored, first_obj) = heapq.heappop()
Here, I'm using - to make it a maxheap. I'm wrapping things in a tuple so that it's sorted by the obj.value, and I'm including the obj as the second value so that I can get it.
Use bisect for binary search
I'm sure you've implemented binary search before. Python has it built in. It even has keyword arguments that you can use to search in only part of the list:
import bisect
insertion_point = bisect.bisect_left(sorted_list, some_item, lo=lo, high=high)
Pay attention to the key argument which is sometimes useful, but may take a little work for it to work the way you want.
namedtuple and dataclasses
Tuples are great, but it can be a pain to deal with remembering the order of the elements or unpacking just a single element in the tuple. That's where namedtuple comes in.
>>> from collections import namedtuple
>>> Point = namedtuple('Point', ['x', 'y'])
>>> p = Point(5, 7)
>>> p
Point(x=5, y=7)
>>> p.x
5
>>> q = p._replace(x=92)
>>> p
Point(x=5, y=7)
>>> q
Point(x=92, y=7)
Keep in mind though that tuples are immutable. If you need something mutable, use a Dataclass instead:
from dataclasses import dataclass
@dataclass
class InventoryItem:
"""Class for keeping track of an item in inventory."""
name: str
unit_price: float
quantity_on_hand: int = 0
def total_cost(self) -> float:
return self.unit_price * self.quantity_on_hand
item = InventoryItem(name='Box', unit_price=19, quantity_on_hand=2)
int, decimal, infinity, etc.
Thankfully, Python's int type supports arbitrarily large values by default
>>> 1 << 128
340282366920938463463374607431768211456
There's also the decimal module if you need to work with things like money where a float isn't accurate enough.
Sometimes, they'll say the range is -2 ^ 32 to 2 ^ 32 - 1. You can get those values via bitshifting:
>>> -(2 ** 32) == -(1 << 32)
True
>>> (2 ** 32) - 1 == (1 << 32) - 1
True
And, sometimes, it's useful to represent infinity via float('inf').
Closures
I'm not sure every interviewer is going to like this, but I tend to skip the OOP stuff and use a bunch of local helper functions so that I can access things via closure:
class Solution(): # This is what LeetCode gave me.
def solveProblem(self, arg1, arg2): # Why they used camelCase, I have no idea.
def helper_function():
# I have access to arg1 and arg2 via closure.
# I don't have to store them on self or pass them around.
return arg1 + arg2
counter = 0
def can_mutate_counter():
# By using nonlocal, I can even mutate counter.
# I rarely use this approach in practice. I usually pass in it
# as an argument and return a value.
nonlocal counter
counter += 1
can_mutate_counter()
return helper_function() + counter
match statement
Did you know Python now has a match statement?
# Taken from: https://learnpython.com/blog/python-match-case-statement/
>>> command = 'Hello, World!'
>>> match command:
... case 'Hello, World!':
... print('Hello to you too!')
... case 'Goodbye, World!':
... print('See you later')
... case other:
... print('No match found')
OrderedDict
If you ever need to implement an LRU cache, it'll be quite helpful to have an OrderedDict.
Python's dicts are now ordered by default. However, the docs for OrderedDict say that there are still some cases where you might need to use OrderedDict. I can't remember. If you never need your dicts to be ordered, just read the docs and figure out if you need an OrderedDict or if you can use just a normal dict.
@functools.cache
If you need a cache, sometimes you can just wrap your code in a function and use @functools.cache:
from functools import cache
@cache
def factorial(n):
return n * factorial(n - 1) if n else 1
Saving memory with the array module
Sometimes you need a really long list of simple numeric (or boolean) values. The array module can help with this, and it's an easy way to decrease your memory usage after you've already gotten your algorithm working.
>>> import array
>>> array_of_bytes = array.array('b')
>>> array_of_bytes.frombytes(b'\0' * (array_of_bytes.itemsize * 10_000_000))
Conclusion
Well, those are my favorite tricks off the top of my head. I'll add more if I think of any.
This is just a single blog post, but if you want more, check out Python 3 Module of the Week.