I've been spending a lot of time practicing on LeetCode recently, so I thought I'd share some of my favorite intermediate-level Python tricks. I'll also cover some newer features of Python you may not have started using yet. I'll start with basic tips and then move to more advanced ones.
Get help()
Python's documentation is pretty great, and some of these examples are taken from there.
For instance, if you just google "heapq", you'll see the official docs for heapq, which are often enough.
However, it's also helpful to sometimes just quickly use help() in the shell. Here, I can't remember that push() is actually called append().
>>> help([])
>>> dir([])
>>> help([].append)
enumerate()
If you need to loop over a list, you can use enumerate() to get both the item as well as the index. As a pneumonic, I like to think for (i, x) in enumerate(...):
for (i, x) in enumerate(some_list):
...
items()
Similarly, you can get both the key and the value at the same time when looping over a dict using items():
for (k, v) in some_dict.items():
...
[] vs. get()
Remember, when you use [] with a dict, if the value doesn't exist, you'll get a KeyError. Rather than see if an item is in the dict and then look up its value, you can use get():
val = some_dict.get(key) # It defaults to None.
if val is None:
...
Similarly, .setdefault() is sometimes helpful.
Some people prefer to just use [] and handle the KeyError since exceptions aren't as expensive in Python as they are in other languages.
range() is smarter than you think
for item in range(items):
...
for index in range(len(items)):
...
# Count by 2s.
for i in range(0, 100, 2):
...
# Count backward from 100 to 0 inclusive.
for i in range(100, -1, -1):
...
# Okay, Mr. Smarty Pants, I'm sure you knew all that, but did you know
# that you can pass a range object around, and it knows how to reverse
# itself via slice notation? :-P
r = range(100)
r = r[::-1] # range(99, -1, -1)
print(f'') debugging
Have you switched to Python's new format strings yet? They're more convenient and safer (from injection vulnerabilities) than % and .format(). They even have a syntax for outputing the thing as well as its value:
# Got 2+2=4
print(f'Got {2+2=}')
for else
Python has a feature that I haven't seen in other programming languages. Both for and while can be followed by an else clause, which is useful when you're searching for something.
for item in some_list:
if is_what_im_looking_for(item):
print(f"Yay! It's {item}.")
break
else:
print("I couldn't find what I was looking for.")
Use a list as a stack
The cost of using a list as a stack is (amortized) O(1):
elements = []
elements.append(element) # Not push
element = elements.pop()
Note that inserting something at the beginning of the list or in the middle is more expensive it has to shift everything to the right--see deque below.
sort() vs. sorted()
# sort() sorts a list in place.
my_list.sort()
# Whereas sorted() returns a sorted *copy* of an iterable:
my_sorted_list = sorted(some_iterable)
And, both of these can take a key function if you need to sort objects.
set and frozenset
Sets are so useful for so many problems! Just in case you didn't know some of these tricks:
# There is now syntax for creating sets.
s = {'Von'}
# There are set "comprehensions" which are like list comprehensions, but for sets.
s2 = {f'{name} the III' for name in s}
{'Von the III'}
# If you can't remember how to use union, intersection, difference, etc.
help(set())
# If you need an immutable set, for instance, to use as a dict key, use frozenset.
frozenset((1, 2, 3))
deque
If you find yourself needing a queue or a list that you can push and pop from either side, use a deque:
>>> from collections import deque
>>>
>>> d = deque()
>>> d.append(3)
>>> d.append(4)
>>> d.appendleft(2)
>>> d.appendleft(1)
>>> d
deque([1, 2, 3, 4])
>>> d.popleft()
1
>>> d.pop()
4
Using a stack instead of recursion
Instead of using recursion (which has a depth of about 1024 frames), you can use a while loop and manually manage a stack yourself. Here's a slightly contrived example:
work = [create_initial_work()]
while work:
work_item = work.pop()
result = process(work_item)
if is_done(result):
return result
work.push(result.pieces[0])
work.push(result.pieces[1])
Using yield from
If you don't know about yield, you can go spend some time learning about that. It's awesome.
Sometimes, when you're in one generator, you need to call another generator. Python now has yield from for that:
def my_generator():
yield 1
yield from some_other_generator()
yield 6
So, here's an example of backtracking:
class Solution:
def problem(self, digits: str) -> List[str]:
def generate_possibilities(work_so_far, remaining_work):
if not remaining_work:
if work_so_far:
yield work_so_far
return
first_part, remaining_part = remaining_work[0], remaining_work[1:]
for i in things_to_try:
yield from generate_possibilities(work_so_far + i, remaining_part)
output = list(generate_possibilities(no_work_so_far, its_all_remaining_work))
return output
This is appropriate if you have less than 1000 "levels" but a ton of possibilities for each of those levels. This won't work if you're going to need more than 1000 layers of recursion. In that case, switch to "Using a stack instead of recursion".
Pre-initialize your list
If you know how long your list is going to be ahead of time, you can avoid needing to resize it multiple times by just pre-initializing it:
dp = [None] * len(items)
collections.Counter()
How many times have you used a dict to count up something? It's built-in in Python:
>>> from collections import Counter
>>> c = Counter('abcabcabcaaa')
>>> c
Counter({'a': 6, 'b': 3, 'c': 3})
defaultdict
Similarly, there's defaultdict:
>>> from collections import defaultdict
>>> d = defaultdict(list)
>>> d['girls'].append('Jocylenn')
>>> d['boys'].append('Greggory')
>>> d
defaultdict(<class 'list'>, {'girls': ['Jocylenn'], 'boys': ['Greggory']})
Notice that I didn't need to set d['girls'] to an empty list before I started appending to it.
heapq
I had heard of heaps in school, but I didn't really know what they were. Well, it turns out they're pretty helpful for several of the problems, and Python has a list-based heap implementation built-in.
If you don't know what a heap is, I recommend this video and this video. They'll explain what a heap is and how to implement one using a list.
The heapq module is a built-in module for managing a heap. It builds on top of an existing list:
import heapq
some_list = ...
heapq.heapify(some_list)
# The head of the heap is some_list[0].
# The len of the heap is still len(some_list).
heapq.heappush(some_list, item)
head_item = heapq.heappop(some_list)
The heapq module also has nlargest and nsmallest built-in so you don't have to implement those things yourself.
Keep in mind that heapq is a minheap. Let's say that what you really want is a maxheap, and you're not working with ints, you're working with objects. Here's how to tweak your data to get it to fit heapq's way of thinking:
heap = []
heapq.heappush(heap, (-obj.value, obj))
(ignored, first_obj) = heapq.heappop()
Here, I'm using - to make it a maxheap. I'm wrapping things in a tuple so that it's sorted by the obj.value, and I'm including the obj as the second value so that I can get it.
Use bisect for binary search
I'm sure you've implemented binary search before. Python has it built-in. It even has keyword arguments that you can use to search in only part of the list:
import bisect
insertion_point = bisect.bisect_left(sorted_list, some_item, lo=lo, high=high)
Pay attention to the key argument which is sometimes useful, but may take a little work for it to work the way you want.
namedtuple and dataclasses
Tuples are great, but it can be a pain to deal with remembering the order of the elements or unpacking just a single element in the tuple. That's where namedtuple comes in.
>>> from collections import namedtuple
>>> Point = namedtuple('Point', ['x', 'y'])
>>> p = Point(5, 7)
>>> p
Point(x=5, y=7)
>>> p.x
5
>>> q = p._replace(x=92)
>>> p
Point(x=5, y=7)
>>> q
Point(x=92, y=7)
Keep in mind that tuples are immutable. I particularly like using namedtuples for backtracking problems. In that case, the immutability is actually a huge asset. I use a namedtuple to represent the state of the problem at each step. I have this much stuff done, this much stuff left to do, this is where I am, etc. At each step, you take the old namedtuple and create a new one in an immutable way.
If you need something mutable, use a dataclass instead:
from dataclasses import dataclass
@dataclass
class InventoryItem:
"""Class for keeping track of an item in inventory."""
name: str
unit_price: float
quantity_on_hand: int = 0
def total_cost(self) -> float:
return self.unit_price * self.quantity_on_hand
item = InventoryItem(name='Box', unit_price=19, quantity_on_hand=2)
dataclasses are great when you want a little class to hold some data, but you don't want to waste much time writing one from scratch.
int, decimal, math.inf, etc.
Thankfully, Python's int type supports arbitrarily large values by default:
>>> 1 << 128
340282366920938463463374607431768211456
There's also the decimal module if you need to work with things like money where a float isn't accurate enough or when you need a lot of decimal places of precision.
Sometimes, they'll say the range is -2 ^ 32 to 2 ^ 32 - 1. You can get those values via bitshifting:
>>> -(2 ** 32) == -(1 << 32)
True
>>> (2 ** 32) - 1 == (1 << 32) - 1
True
Sometimes, it's useful to initialize a variable with math.inf (i.e. infinity) and then try to find new values less than that.
Closures
I'm not sure every interviewer is going to like this, but I tend to skip the OOP stuff and use a bunch of local helper functions so that I can access things via closure:
class Solution(): # This is what LeetCode gave me.
def solveProblem(self, arg1, arg2): # Why they used camelCase, I have no idea.
def helper_function():
# I have access to arg1 and arg2 via closure.
# I don't have to store them on self or pass them around
# explicitly.
return arg1 + arg2
counter = 0
def can_mutate_counter():
# By using nonlocal, I can even mutate counter.
# I rarely use this approach in practice. I usually pass in it
# as an argument and return a value.
nonlocal counter
counter += 1
can_mutate_counter()
return helper_function() + counter
match statement
Did you know Python now has a match statement?
# Taken from: https://learnpython.com/blog/python-match-case-statement/
>>> command = 'Hello, World!'
>>> match command:
... case 'Hello, World!':
... print('Hello to you too!')
... case 'Goodbye, World!':
... print('See you later')
... case other:
... print('No match found')
It's actually much more sophisticated than a switch statement, so take a look, especially if you've never used match in a functional language like Haskell.
OrderedDict
If you ever need to implement an LRU cache, it'll be quite helpful to have an OrderedDict.
Python's dicts are now ordered by default. However, the docs for OrderedDict say that there are still some cases where you might need to use OrderedDict. I can't remember. If you never need your dicts to be ordered, just read the docs and figure out if you need an OrderedDict or if you can use just a normal dict.
@functools.cache
If you need a cache, sometimes you can just wrap your code in a function and use functools.cache:
from functools import cache
@cache
def factorial(n):
return n * factorial(n - 1) if n else 1
print(factorial(5))
...
factorial.cache_info() # CacheInfo(hits=3, misses=8, maxsize=32, currsize=8)
Debugging ListNodes
A lot of the problems involve a ListNode class that's provided by LeetCode. It's not very "debuggable". Add this code temporarily to improve that:
def list_node_str(head):
seen_before = set()
pieces = []
p = head
while p is not None:
if p in seen_before:
pieces.append(f'loop at {p.val}')
break
pieces.append(str(p.val))
seen_before.add(p)
p = p.next
joined_pieces = ', '.join(pieces)
return f'[{joined_pieces}]'
ListNode.__str__ = list_node_str
Saving memory with the array module
Sometimes you need a really long list of simple numeric (or boolean) values. The array module can help with this, and it's an easy way to decrease your memory usage after you've already gotten your algorithm working.
>>> import array
>>> array_of_bytes = array.array('b')
>>> array_of_bytes.frombytes(b'\0' * (array_of_bytes.itemsize * 10_000_000))
Pay close attention to the type of values you configure the array to accept. Read the docs.
I'm sure there's a way to use individual bits for an array of booleans to save even more space, but it'd probably cost more CPU, and I generally care about CPU more than memory.
Using an exception for the success case rather than the error case
A lot of Python programmers don't like this trick because it's equivalent to goto, but I still occasionally find it convenient:
class Eureka(StopIteration):
"""Eureka means "I found it!" """
pass
def do_something_else():
some_value = 5
raise Eureka(some_value)
def do_something():
do_something_else()
try:
do_something()
except Eureka as exc:
print(f'I found it: {exc.args[0]}')
Using VS Code, etc.
VS Code has a pretty nice Python extension. If you highlight the code and hit shift-enter, it'll run it in a shell. That's more convenient than just typing everything directly in the shell. Other editors have something similar, or perhaps you use a Jupyter notebook for this.
Another thing that helps me is that I'll often have separate files open with separate attempts at a solution. I guess you can call this the "fast" approach to branching.
Conclusion
Well, those are my favorite tricks off the top of my head. I'll add more if I think of any.
This is just a single blog post, but if you want more, check out Python 3 Module of the Week.