In this quiz, you’ll test your understanding of Dictionaries in Python.
By working through this quiz, you’ll revisit how to create dictionaries using literals and the dict() constructor, how to use Python’s operators and built-in functions to manipulate them, and how they’re implemented as a hash map for fast key lookups.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
In this quiz, you’ll test your understanding of Instance, Class, and Static Methods in Python. By working through this quiz, you’ll revisit the differences between these methods and how to use them effectively in your Python code.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
In this quiz, you’ll test your understanding of Python Set Comprehensions: How and When to Use Them.
Set comprehensions are a concise and quick way to create, transform, and filter sets in Python.
They can significantly enhance your code’s conciseness and readability compared to using regular for loops to process your sets.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
In this quiz, you’ll test your understanding of Continuous Integration and Deployment for Python With GitHub Actions.
By working through this quiz, you’ll revisit how to use GitHub Actions and workflows to automate linting, testing, and deployment of a Python project. You’ll also review how to secure your credentials and automate security and dependency updates.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
The DuckDB database provides a seamless way to handle large datasets in Python with Online Analytical Processing (OLAP) optimization. You can create databases, verify data imports, and perform efficient data queries using both SQL and DuckDB’s Python API.
By the end of this tutorial, you’ll understand that:
.df() or .pl() methods.The tutorial will equip you with the practical knowledge necessary to get started with DuckDB, including its Online Analytical Processing (OLAP) features, which enable fast access to data through query optimization and buffering.
Ideally, you should already have a basic understanding of SQL, particularly how its SELECT keyword can be used to read data from a relational database. However, the SQL language is very user-friendly, and the examples used here are self-explanatory.
Now, it’s time for you to start learning why there’s a growing buzz surrounding DuckDB.
Get Your Code:Click here to download the free sample code that shows you how to use DuckDB in Python.
Take the Quiz: Test your knowledge with our interactive “Introducing DuckDB” quiz. You’ll receive a score upon completion to help you track your learning progress:
Interactive Quiz
Introducing DuckDBThis quiz will challenge your knowledge of working with DuckDB. You won't find all the answers in the tutorial, so you'll need to do some extra investigation. By finding all the answers, you're sure to learn some interesting things along the way.
To use DuckDB, you first need to install it. Fortunately, DuckDB is self-contained, meaning it won’t interfere with your existing Python environment.
You use python -m pip install duckdb to install it from the command prompt. If you’re working in a Jupyter Notebook, the command becomes !python -m pip install duckdb. The supporting downloadable code for this tutorial is also presented in a Jupyter Notebook.
Once the installation is complete, you can quickly test your installation with a query:
>>> importduckdb>>> duckdb.sql("SELECT 'whistling_duck' AS waterfowl, 'whistle' AS call")┌────────────────┬─────────┐│ waterfowl │ call ││ varchar │ varchar │├────────────────┼─────────┤│ whistling_duck │ whistle │└────────────────┴─────────┘To test that everything works, you first import the duckdb library before running a test SQL query. In SQL, a query is a command you use to interact with the data in your database. You commonly use queries to view, add, update, and delete your data.
In this example, you write a SQL SELECT statement to view some data defined by the query. By passing it to the sql() function, you run the query and produce the result shown.
Your query creates a table with two columns named waterfowl and call. These contain the data "whistling_duck" and "whistle", respectively. The data types of both columns are varchar, which is the data type DuckDB uses to store variable-length character strings. Running your query using duckdb.sql() uses the default in-memory database. This means that the data are temporary and will disappear when you end your Python session.
If you see the output shown above, your installation is working perfectly.
Note: DuckDB queries are not case-sensitive. However, writing reserved SQL keywords in uppercase is standard practice. Also, a terminating semicolon (;) is optional in SQL and isn’t used in this tutorial, though you may encounter it elsewhere.
Now that you know how to set things up, it’s time to dive into some of the features that make DuckDB easy to use. In the next section, you’ll create a database table using data imported from an existing file. You’ll also learn how to check that the data has been imported correctly.
While it’s possible to create database tables using SQL, it’s more common to read data from an external file, perhaps one containing data you’ve extracted from another system, and allow DuckDB to create and populate the table.
DuckDB supports reading from and writing to a range of common file types such as Parquet, CSV, and JSON. In this example, you’ll use data stored in the presidents.parquetParquet file included in your downloadable materials to create a table.
The presidents.parquet file contains the following six fields:
| Heading | Meaning | Data Type |
|---|---|---|
sequence | Order of presidency | int64 |
last_name | President’s last name | varchar |
first_name | President’s first name | varchar |
term_start | Start of presidency term | date |
term_end | End of presidency term | date |
party_id | Number representing political party | int64 |
When you import data, it gets placed into a DuckDBPyRelation object. In DuckDB, a relation stores a query definition but not its data. To see the data your relation represents, you must do so interactively by viewing it or running an SQL query against it to see specific data.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Over the years I have encountered my share of memorable problems. They were remarkably complex, hard to debug, completely obvious in retrospect, or plain funny. This is the story of one of them.
At the beginning, there was a suite of automated tests that I was maintaining. One day one of them failed. Not a big deal, unfortunate reality is that some of them fail sometimes for various reasons. Usually they pass when run again and we can blame unreliable infrastructure, transient networking issue or misalignment of the stars. But few days later the same test failed again. And then again. It was clear that there’s something going on and this particular test is intermittently failing. I had to figure out what is happening and how can I make the test provide the same result reliably.
(Note the choice of words here. My goal was not to make the test passing, or “green”. There might as well have been a bug in the test itself, or in the product. At this point nobody knew. The main goal was understanding the issue and making sure test is reliably providing the same result - whether it is pass or fail.)
Before we move on, there’s some relevant context that I need to share. That suite contained only UI tests. Running them all took about an hour. They were running against staging environment few times a day. The test that was failing was responsible for checking a chart which plots the data from last 30 days. There were other tests verifying other charts, sometimes using different time spans. The website used the same generic chart component in all cases. These other tests never failed.
On a high level, the failing test consisted of three main steps: request the data from last 30 days using the API, read the data from the graph on the website, and compare both. Test was considered failed if there was any difference between the data from these two sources. Python deepdiff package was used for comparison. To make it possible, data from API was locally transformed to mimic the structure returned by function responsible for reading the data from UI.
Testing infrastructure had few distinct pieces. There was a Jenkins server that triggered a test suite run at certain times of the day. Job executors were containers in a Kubernetes cluster. To facilitate UI testing, there was a Selenium Grid server with few workers hosted as virtual machines on OpenStack. Tests were running against staging environment of the product, which was also hosted on a Kubernetes cluster, but different than the one where job executors were. I believe all that was scattered across two data centers, with most of testing infrastructure being co-located, and product under test being elsewhere.
Now, let’s get back to the story.
The very first thing I did was looking into test logs. Unfortunately, differences between objects as reported by deepdiff in this particular case are not easy to read (see below). The amount of data is overwhelming, and displaying everything in single line contributes to the challenge. The log made it clear that lists returned by API and read from UI are different, but it was not immediately obvious where exactly these differences are.
> assert not deepdiff.DeepDiff(expected_graph_data, actual_graph_data)E assert not {'values_changed': {"root[0]['Date']": {'new_value': '1970-01-01', 'old_value': '1970-01-02'}, "root[0]['Foo']": {'new_value': 46, 'old_value': 23}, "root[0]['Bar']": {'new_value': 60, 'old_value': 99}, "root[0]['Total']": {'new_value': 106, 'old_value': 122}, "root[1]['Date']": {'new_value': '1970-01-02', 'old_value': '1970-01-03'}, "root[1]['Foo']": {'new_value': 23, 'old_value': 26}, "root[1]['Bar']": {'new_value': 99, 'old_value': 92}, "root[1]['Total']": {'new_value': 122, 'old_value': 118}, "root[2]['Date']": {'new_value': '1970-01-03', 'old_value': '1970-01-04'}, "root[2]['Foo']": {'new_value': 26, 'old_value': 49}, "root[2]['Bar']": {'new_value': 92, 'old_value': 86}, "root[2]['Total']": {'new_value': 118, 'old_value': 135}, "root[3]['Date']": {'new_value': '1970-01-04', 'old_value': '1970-01-05'}, "root[3]['Foo']": {'new_value': 49, 'old_value': 68}, "root[3]['Bar']": {'new_value': 86, 'old_value': 60}, "root[3]['Total']": {'new_value': 135, 'old_value': 128}, "root[4]['Date']": {'new_value': '1970-01-05', 'old_value': '1970-01-06'}, "root[4]['Foo']": {'new_value': 68, 'old_value': 33}, "root[4]['Bar']": {'new_value': 60, 'old_value': 14}, "root[4]['Total']": {'new_value...ue': 25}, "root[24]['Bar']": {'new_value': 29, 'old_value': 78}, "root[24]['Total']": {'new_value': 106, 'old_value': 103}, "root[25]['Date']": {'new_value': '1970-01-26', 'old_value': '1970-01-27'}, "root[25]['Foo']": {'new_value': 25, 'old_value': 57}, "root[25]['Bar']": {'new_value': 78, 'old_value': 84}, "root[25]['Total']": {'new_value': 103, 'old_value': 141}, "root[26]['Date']": {'new_value': '1970-01-27', 'old_value': '1970-01-28'}, "root[26]['Foo']": {'new_value': 57, 'old_value': 48}, "root[26]['Bar']": {'new_value': 84, 'old_value': 18}, "root[26]['Total']": {'new_value': 141, 'old_value': 66}, "root[27]['Date']": {'new_value': '1970-01-28', 'old_value': '1970-01-29'}, "root[27]['Foo']": {'new_value': 48, 'old_value': 89}, "root[27]['Bar']": {'new_value': 18, 'old_value': 14}, "root[27]['Total']": {'new_value': 66, 'old_value': 103}, "root[28]['Date']": {'new_value': '1970-01-29', 'old_value': '1970-01-30'}, "root[28]['Foo']": {'new_value': 89, 'old_value': 61}, "root[28]['Bar']": {'new_value': 14, 'old_value': 66}, "root[28]['Total']": {'new_value': 103, 'old_value': 127}}, 'iterable_item_added': {'root[29]': {'Date': '1970-01-30', 'Foo': 61, 'Bar': 66, 'Total': 127}}}Trying to understand this log felt daunting, so my next step was running the failing test locally, in isolation. Predictably, it passed. I didn’t have the high hopes that I will be able to reproduce the problem right away, but that was a cheap thing to try, so I think it was worth giving a shot.
At this point I decided there is no way around it and I have to better understand how API and UI responses are different. I copied the log line into editor and inserted a new line character after each },. Few more changes later I had a form that was a little easier to decipher.
Deepdiff shows the differences between elements under the same index in lists. But focusing on elements with the same date value revealed that they are fundamentally the same. Values appearing under “old_value” in one list appears as “new_value” in the other list, just under different index. I have put colored overlay on the screenshot below to make it easier to see. You can think of these lists as mostly the same, but one is shifted when compared to other; or you can say that one list has extra element added at the end, while the other has extra element added at the very beginning. Specifically, API read data from January 2nd to February 1st, but UI displayed data from January 1st to January 31st. There’s a large overlap, but deepdiff output obscured this key insight.
At this point I had an idea what is wrong, but I had no clue why, and why it would affect only this one single test. So in the next step I wanted to see if there are any patterns to the failure. I grabbed test results from last few weeks and put them in the spreadsheet. I added columns for basic things, like the result itself, how long did it take for test to finish, date and time when test was run. To make failing tests visually distinct, I added background color to highlight them. In separate column I tagged all rows where test was running for a first time in a given day. Then I added columns representing known issues that we encountered in previous few weeks, to see if all failures fall into one of them.
While there wasn’t a clear and predictable pattern, I did notice a curious thing - if the test failed, it would fail in the first run of a given day. Subsequent runs of any day never failed. And the first run in a day always started shortly after midnight UTC.
That allowed me to construct a working hypothesis: the issue is somehow related to time and there’s only a short window when it may occur, maybe up to few hours. That window is located around midnight UTC. Such hypothesis explains why subsequent pipeline runs never failed, and why I was never successful at reproducing the issue locally - I am located east of UTC line and I would have to try running the test way outside of working hours. Of course I didn’t know if I was up to something or I was just creating complex ad hoc hypothesis that fits the data. But it directed my next step.
To corroborate the hypothesis I needed some new information, things I didn’t have before. To gather it, I have added further logging in the test suite. First, I have used Selenium JavaScript execution capability to obtain the date and time as the browser “sees” it. Then I have done the same from Python, which both drives Selenium and requests data from API. The important part is that Python code is executed directly on test runner (container in Kubernetes) and JavaScript code is executed in the browser (Selenium Grid VM on OpenStack).
diff --git package/tests/ui/test_failing.py package/tests/ui/test_failing.pyindex 1234567..abcdef0 100644--- package/tests/ui/test_failing.py+++ package/tests/ui/test_failing.py@@ -10,6 +10,13 @@ def test_failing_test(user_app, some_fixture):"""
view = navigate_to(user_app.some_app, "SomeViewName")
+ browser_time_string = view.browser.execute_script("return new Date().toTimeString()")+ browser_utc_string = view.browser.execute_script("return new Date().toUTCString()")+ view.logger.info(+ "[JavaScript] Time right now: %s ; UTC time: %s",+ browser_time_string,+ browser_utc_string,+ ) expected_x_axis = get_xaxis_values()
view.items.item_select(some_value)
view.graph.wait_displayed()
diff --git package/utils/utils.py package/utils/utils.pyindex 1234567..abcdef0 100644--- package/utils/utils.py+++ package/utils/utils.py@@ -10,6 +10,14 @@ METRIC_MAP = {def _get_dates_range(some_param="Daily", date=None):
+ current_time = arrow.now()+ log.info(+ "[Python] Time right now: %s ; TZ name: %s ; TZ offset: %s ; UTC time: %s",+ current_time,+ current_time.strftime("%Z"),+ current_time.strftime("%z"),+ arrow.utcnow(),+ ) try:
date = arrow.get(date)
except TypeError:
With the above patch applied and deployed, all I needed to do was waiting for the next failure. I hoped that new logs would reveal some more information once it fails again.
That turned out to be true. JavaScript showed a date one day earlier than Python. In fact, the time in JavaScript was about 15 minutes earlier than in Python. So if test suite ran around midnight, and we got to offending test within 15 minutes of suite start, then Python would request data through API for some dates, but website in browser would think it is still the previous day, and request different set of dates. It means that the window where issue occurs is extremely small - just around 15 minutes each day.
[JavaScript] Time right now: Thu Jan 01 1970 23:58:17 GMT+0000 (Coordinated Universal Time) ; UTC time: Thu, 01 Jan 1970 23:58:17GMT[Python] Time right now: 1970-01-02T00:14:36.042473+00:00 ; TZ name: UTC ; TZ offset: +0000 ; UTC time: 1970-01-02T00:14:36.042473+00:00This concludes the main part of the debugging story - at this point we knew what is wrong, we knew that failure is not caused by a bug in a test or a product, and it was clear that the solution is for all machines involved in testing to reconcile date and time. It also seemed like the JavaScript shows wrong date, which might mean that the issue is with Selenium Grid machines or OpenStack instance.
I connected to all Selenium Grid machines using SSH and checked their local time using date command. They were about 15 minutes behind their wall-clock time. I assumed the difference is caused by various OpenStack and underlying infrastructure maintenance work, so I just used hwclock to force OS clock to synchronize with hardware clock and moved on with my day.
Couple of days later I connected to these machines again and noticed that the local time is behind again, but only by about 15 seconds. It looked like the local clock is drifting by about 5 seconds a day. It might not sound like much, but it also meant that it’s only a matter of time before original issue happens again. Clearly someone logging in to these machines every once in a while and resetting clock would not be a good long term solution - we needed something that can automatically keep time synchronized.
That something is called NTP and all the machines already had chrony installed. However, it didn’t seem to work correctly. While the commands succeeded and logs did not show any problems, the clock would just not change. After few frustrating hours I think I ruled out all possible causes at the operating system level and came to the conclusion that perhaps the NTP traffic to public servers is blocked by data center firewall. I reached out to OpenStack administrators for help and they told me that there is a blessed NTP server instance inside the data center that I should use. Once I configured chrony to use it as a source, everything finally worked.
This way browsers started to consistently report the same time as Python executors. That fixed the original issue and we did not observe any test failures caused by it again.
You can check whether iterables contain the same elements in Python with equality checks, type conversions, sets, Counter, or looping helpers.
Table of contents
If we have two lists and we wanted to know whether the items in these two lists are the same, we could use the equality operator (==):
>>> lines1=["Grains","Kindred","Zia"]>>> lines2=["Grains","Kindred","Zia"]>>> lines1==lines2TrueThe same thing works for comparing tuples:
>>> p=(3,4,8)>>> q=(3,5,7)>>> p==qFalseBut what if we wanted to compare a list and a tuple?
We can't use a simple equality check for that:
>>> lines1=["Grains","Kindred","Zia"]>>> lines2=("Grains","Kindred","Zia")>>> lines1==lines2FalseTo compare the items in …
Some notes on implementing attention blocks in pure Python + Numpy. The focus here is on the exact implementation in code, explaining all the shapes throughout the process. The motivation for why attention works is not covered here - there are plenty of excellent online resources explaining it.
Several papers are mentioned throughout the code; they are:
We'll start with the most basic scaled dot product self-attention, working on a single sequence of tokens, without masking.
The input is a 2D array of shape (N, D). N is the length of the sequence (how many tokens it contains) and D is the embedding depth - the length of the embedding vector representing each token [1]. D could be something like 512, or more, depending on the model.
A self-attention module is parameterized with three weight matrices, Wk, Wq and Wv. Some variants also have accompanying bias vectors, but the AIAYN paper doesn't use them, so I'll skip them here. In the general case, the shape of each weight matrix is (D, HS), where HS is some fraction of D. HS stands for "head size" and we'll see what this means soon. This is a diagram of a self-attention module (the diagram assumes N=6, D is some large number and so is HS). In the diagram, @ stands for matrix multiplication (Python/Numpy syntax):
Here's a basic Numpy implementation of this:
# self_attention the way it happens in the Transformer model. No bias.# D = model dimension/depth (length of embedding)# N = input sequence length# HS = head size## x is the input (N, D), each token in a row.# Each of W* is a weight matrix of shape (D, HS)# The result is (N, HS)defself_attention(x,Wk,Wq,Wv):# Each of these is (N, D) @ (D, HS) = (N, HS)q=x@Wqk=x@Wkv=x@Wv# kq: (N, N) matrix of dot products between each pair of q and k vectors.# The division by sqrt(HS) is the scaling.kq=q@k.T/np.sqrt(k.shape[1])# att: (N, N) attention matrix. The rows become the weights that sum# to 1 for each output vector.att=softmax_lastdim(kq)returnatt@v# (N, HS)The "scaled" part is just dividing kq by the square root of HS, which is done to keep the values of the dot products manageable (otherwise they would grow with the size of the contracted dimension).
The only dependency is a function for calculating Softmax across the last dimension of an input array:
defsoftmax_lastdim(x):"""Compute softmax across last dimension of x. x is an arbitrary array with at least two dimensions. The returned array has the same shape as x, but its elements sum up to 1 across the last dimension."""# Subtract the max for numerical stabilityex=np.exp(x-np.max(x,axis=-1,keepdims=True))# Divide by sums across last dimensionreturnex/np.sum(ex,axis=-1,keepdims=True)When the input is 2D, the "last dimension" is the columns. Colloquially, this Softmax function acts on each row of x separately; it applies the Softmax formula to the elements (columns) of the row, ending up with a row of numbers in the range [0,1] that all sum up to 1.
Another note on the dimensions: it's possible for the Wv matrix to have a different second dimension from Wq and Wk. If you look at the diagram, you can see this will work out, since the softmax produces (N, N), and whatever the second dimension of V is, will be the second dimension of the output. The AIAYN paper designates these dimensions as d_k and d_v, but in practice d_k=d_v in all the variants it lists. I found that these dimensions are typically the same in other papers as well. Therefore, for simplicity I just made them all equal to D in this post; if desired, a variant with different d_k and d_v is a fairly trivial modification to this code.
In the real world, the input array is unlikely to be 2D because models are trained on batches of input sequences. To leverage the parallelism of modern hardware, whole batches are typically processed in the same operation.
The batched version of scaled self-attention is very similar to the non-batched one, due to the magic of Numpy matrix multiplication and broadcasts. Now the input shape is (B, N, D), where B is the batch dimension. The W* matrices are still (D, HS); multiplying a (B, N, D) array by (D, HS) performs contraction between the last axis of the first array and the first axis of the second array, resulting in (B, N, HS). Here's the code, with the dimensions annotated for each operation:
# self_attention with inputs that have a batch dimension.# x has shape (B, N, D)# Each of W* has shape (D, D)defself_attention_batched(x,Wk,Wq,Wv):q=x@Wq# (B, N, HS)k=x@Wk# (B, N, HS)v=x@Wv# (B, N, HS)kq=q@k.swapaxes(-2,-1)/np.sqrt(k.shape[-1])# (B, N, N)att=softmax_lastdim(kq)# (B, N, N)returnatt@v# (B, N, HS)Note that the only difference between this and the non-batched version is the line calculating kq:
In fact, this function could also calculate the non-batched version! From now on, we'll assume that all inputs are batched, and all operations are implicitly batched. I'm not going to be using the "batched" prefix or suffix on functions any more.
The basic underlying idea of the attention module is to shift around the multi-dimensional representations of tokens in the sequence towards a better representation of the entire sequence. The tokens attend to each other. Specifically, the matrix produced by the Softmax operation is called the attention matrix. It's (N, N); for each token it specifies how much information from every other token in the sequence should be taken into account. For example, a higher number in cell (R, C) means that there's a stronger relation of token at index R in the sequence to the token at index C.
Here's a nice example from the AIAYN paper, showing a word sequence and the weights produced by two attention heads (purple and brown) for a given position in the input sequence:
This shows how the model is learning to resolve what the word "its" refers to in the sentence. Let's take just the purple head as an example. The index of token "its" in the sequence is 8, and the index of "Law" is 1. In the attention matrix for this head, the value at index (8, 1) will be very high (close to 1), with other values in the same row much lower.
While this intuitive explanation isn't critical to understand how attention is implemented, it will become more important when we talk about masked self-attention later on.
The attention mechanism we've seen so far has a single set of K, Q and V matrices. This is called one "head" of attention. In today's models, there are typically multiple heads. Each head does its attention job separately, and in the end all these results are concatenated and feed through a linear layer.
In what follows, NH is the number of heads and HS is the head size. Typically, NH times HS would be D; for example, the AIAYN paper mentions several configurations for D=512: NH=8 and HS=64, NH=32 and HS=16, and so on [2]. However, the math works out even if this isn't the case, because the final linear ("projection") layer maps the output back to (N, D).
Assuming the previous diagram showing a self-attention module is a single head with input (N, D) and output (N, HS), this is how multiple heads are combined:
Each of the (NH) heads has its own parameter weights for Q, K and V. Each attention head outputs a (N, HS) matrix; these are concatenated along the last dimension to (N, NH * HS), which is passed through a final linear projection.
Here's a function implementing (batched) multi-head attention; for now, please ignore the code inside do_mask conditions:
# x has shape (B, N, D)# In what follows:# NH = number of heads# HS = head size# Each W*s is a list of NH weight matrices of shape (D, HS).# Wp is a weight matrix for the final linear projection, of shape (NH * HS, D)# The result is (B, N, D)# If do_mask is True, each attention head is masked from attending to future# tokens.defmultihead_attention_list(x,Wqs,Wks,Wvs,Wp,do_mask=False):# Check shapes.NH=len(Wks)HS=Wks[0].shape[1]assertlen(Wks)==len(Wqs)==len(Wvs)forWinWqs+Wks+Wvs:assertW.shape[1]==HSassertWp.shape[0]==NH*HS# List of head outputshead_outs=[]ifdo_mask:# mask is a lower-triangular (N, N) matrix, with zeros above# the diagonal and ones on the diagonal and below.N=x.shape[1]mask=np.tril(np.ones((N,N)))forWk,Wq,Wvinzip(Wks,Wqs,Wvs):# Calculate self attention for each head separatelyq=x@Wq# (B, N, HS)k=x@Wk# (B, N, HS)v=x@Wv# (B, N, HS)kq=q@k.swapaxes(-2,-1)/np.sqrt(k.shape[-1])# (B, N, N)ifdo_mask:# Set the masked positions to -inf, to ensure that a token isn't# affected by tokens that come after it in the softmax.kq=np.where(mask==0,-np.inf,kq)att=softmax_lastdim(kq)# (B, N, N)head_outs.append(att@v)# (B, N, HS)# Concatenate the head outputs and apply the final linear projectionall_heads=np.concatenate(head_outs,axis=-1)# (B, N, NH * HS)returnall_heads@Wp# (B, N, D)It is possible to vectorize this code even further; you'll sometimes see the heads laid out in a separate (4th) dimension instead of being a list. See the Vectorizing across the heads dimension section.
Attention modules can be used in both encoder and decoder blocks. Encoder blocks are useful for things like language understanding or translation; for these, it makes sense for each token to attend to all the other tokens in the sequence.
However, for generative models this presents a problem: if during training a word attends to future words, the model will just "cheat" and not really learn how to generate the next word from only past words. This is done in a decoder block, and for this we need to add masking to attention.
Conceptually, masking is very simple. Consider the sentence:
People like watching funny cat videos
When our attention code generates the att matrix, it's a square (N, N) matrix with attention weights from each token to each other token in the sequence:
What we want is for all the gray cells in this matrix to be zero, to ensure that a token doesn't attend to future tokens. The blue cells in the matrix add up to 1 in each row, after the softmax operation.
Now take a look at the previous code sample and see what happens when do_mask=True:
Another name for masked self-attention is causal self-attention. This is a very good name that comes from causal systems in control theory.
So far we've been working with self-attention blocks, where the self suggests that elements in the input sequence attend to other elements in the same input sequence.
Another variant of attention is cross-attention, where elements of one sequence attend to elements in another sequence. This variant exists in the decoder block of the AIAYN paper. This is a single head of cross-attention:
Here we have two sequences with potentially different lengths: xq and xv. xq is used for the query part of attention, while xv is used for the key and value parts. The rest of the dimensions remain as before. The output of such a block is shaped (Nq, HS).
This is an implementation of multi-head cross-attention; it doesn't include masking, since masking is not typically necessary in cross attention - it's OK for elements of xq to attend to all elements of xv[3]:
# Cross attention between two input sequences that can have different lengths.# xq has shape (B, Nq, D)# xv has shape (B, Nv, D)# In what follows:# NH = number of heads# HS = head size# Each W*s is a list of NH weight matrices of shape (D, HS).# Wp is a weight matrix for the final linear projection, of shape (NH * HS, D)# The result is (B, Nq, D)defmultihead_cross_attention_list(xq,xv,Wqs,Wks,Wvs,Wp):# Check shapes.NH=len(Wks)HS=Wks[0].shape[1]assertlen(Wks)==len(Wqs)==len(Wvs)forWinWqs+Wks+Wvs:assertW.shape[1]==HSassertWp.shape[0]==NH*HS# List of head outputshead_outs=[]forWk,Wq,Wvinzip(Wks,Wqs,Wvs):q=xq@Wq# (B, Nq, HS)k=xv@Wk# (B, Nv, HS)v=xv@Wv# (B, Nv, HS)kq=q@k.swapaxes(-2,-1)/np.sqrt(k.shape[-1])# (B, Nq, Nv)att=softmax_lastdim(kq)# (B, Nq, Nv)head_outs.append(att@v)# (B, Nq, HS)# Concatenate the head outputs and apply the final linear projectionall_heads=np.concatenate(head_outs,axis=-1)# (B, Nq, NH * HS)returnall_heads@Wp# (B, Nq, D)The multihead_attention_list implementation shown above uses lists of weight matrices as input. While this makes the code clearer, it's not a particularly friendly format for an optimized implementation - especially on accelerators like GPUs and TPUs. We can vectorize it further by creating a new dimension for attention heads.
To understand the trick being used, consider a basic matmul of (8, 6) by (6, 2):
Now suppose we want to multiply our LHS by another (6, 2) matrix. We can do it all in the same operation by concatenating the two RHS matrices along columns:
If the yellow RHS block in both diagrams is identical, the green block of the result will be as well. And the violet block is just the matmul of the LHS by the red block of the RHS. This stems from the semantics of matrix multiplication, and is easy to verify on paper.
Now back to our multi-head attention. Note that we multiply the input x by a whole list of weight matrices - in fact, by three lists (one list for Q, one for K, and another for V). We can use the same vectorization technique by concatenating all these weight matrices into a single one. Assuming that NH * HS = D, the shape of the combined matrix is (D, 3 * D). Here's the vectorized implementation:
# x has shape (B, N, D)# In what follows:# NH = number of heads# HS = head size# NH * HS = D# W is expected to have shape (D, 3 * D), with all the weight matrices for# Qs, Ks, and Vs concatenated along the last dimension, in this order.# Wp is a weight matrix for the final linear projection, of shape (D, D).# The result is (B, N, D).# If do_mask is True, each attention head is masked from attending to future# tokens.defmultihead_attention_vec(x,W,NH,Wp,do_mask=False):B,N,D=x.shapeassertW.shape==(D,3*D)qkv=x@W# (B, N, 3 * D)q,k,v=np.split(qkv,3,axis=-1)# (B, N, D) eachifdo_mask:# mask is a lower-triangular (N, N) matrix, with zeros above# the diagonal and ones on the diagonal and below.mask=np.tril(np.ones((N,N)))HS=D//NHq=q.reshape(B,N,NH,HS).transpose(0,2,1,3)# (B, NH, N, HS)k=k.reshape(B,N,NH,HS).transpose(0,2,1,3)# (B, NH, N, HS)v=v.reshape(B,N,NH,HS).transpose(0,2,1,3)# (B, NH, N, HS)kq=q@k.swapaxes(-1,-2)/np.sqrt(k.shape[-1])# (B, NH, N, N)ifdo_mask:# Set the masked positions to -inf, to ensure that a token isn't# affected by tokens that come after it in the softmax.kq=np.where(mask==0,-np.inf,kq)att=softmax_lastdim(kq)# (B, NH, N, N)out=att@v# (B, NH, N, HS)returnout.transpose(0,2,1,3).reshape(B,N,D)@Wp# (B, N, D)This code computes Q, K and V in a single matmul, and then splits them into separate arrays (note that on accelerators these splits and later transposes may be very cheap or even free as they represent a different access pattern into the same data).
Each of Q, K and V is initially (B, N, D), so they are reshaped into a more convenient shape by first splitting the D into (NH, HS), and finally changing the order of dimensions to get (B, NH, N, HS). In this format, both B and NH are considered batch dimensions that are fully parallelizable. The QK^T computation can then proceed as before, and Numpy will automatically perform the matmul over all the batch dimensions.
Sometimes you'll see an alternative notation used in papers for these matrix multiplications: numpy.einsum. For example, in our last code sample the computation of kq could also be written as:
kq=np.einsum("bhqd,bhkd->bhqk",q,k)/np.sqrt(k.shape[-1])The full code for these samples, with tests, is available in this repository.
| [1] | In LLM papers, D is often called d_{model}. |
| [2] | In the GPT-3 paper, this is also true for all model variants. For example, the largest 175B model has NH=96, HS=128 and D=12288. |
| [3] | It's also not as easy to define mathematically: how do we make a non-square matrix triangular? And what does it mean when the lengths of the two inputs are different? |
Three years ago I shared the As-Any Hack on this blog. That hack is a way on stable rust to get upcasting to supertraits working in Rust. To refresh your memory, the goal was to make something like this work:
#[derive(Debug)]structAnyBox(Box<dynDebugAny>);traitDebugAny: Any+Debug{}impl<T: Any+Debug+'static>DebugAnyforT{}The problem? Even though DebugAny inherits from Any, Rust wouldn't let you use methods from Any on a dyn DebugAny. So while you could call DebugAny methods just fine, trying to use downcast_ref from Any (the reason to use Any in the first place) would fail:
fnmain(){letany_box=AnyBox(Box::new(42i32));dbg!(any_box.0.downcast_ref::<i32>());// Compile error}The same would happen if we tried to cast it into an &dyn Any? A compile error again:
fnmain(){letany_box=AnyBox(Box::new(42i32));letany=&*any_box.0as&dynAny;dbg!(any.downcast_ref::<i32>());}But there is good news! As of Rust 1.86, this is finally fixed. The cast now works:
[src/main.rs:14:5] any.downcast_ref::<i32>() = Some(
42,
)
At the time of writing, this fix is in the beta channel, but stable release is just around the corner. That means a lot of old hacks can finally be retired. At least once your MSRV moves up.
Thank you so much to everyone who worked on this to make it work!
If you signed up for an account on PythonAnywhere after 25 March 2025, you’ll have Python versions 3.11, 3.12 and 3.13 available. Additionally, the underlying operating system for your account will be Ubuntu 22.04, rather than the 20.04 used by older accounts.
If you signed up before that date, you’ll be on an older “system image” – essentially the version of the operating system and the set of installed packages that you have access to. You can switch to the new system image from the “Account” page, but you may need to make changes to your code and/or virtualenvs to make everything work – there’s more information on that page.
This post has more details on what’s new in the “innit” system image. There’s a lot!
pytest-html has got to be one of my all time favorite plugins.
pytest-html is a plugin for pytest that generates a HTML report for test results.
This episode digs into some of the super coolness of pytest-html.
Sponsored by:
In this quiz, you’ll test your understanding of Using Python’s .__dict__ to Work With Attributes.
By working through this quiz, you’ll revisit how .__dict__ holds an object’s writable attributes, allowing for dynamic manipulation and introspection. You’ll also review how both vars() and .__dict__ let you inspect an object’s attributes, and the common use cases of .__dict__ in Python applications.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
I recently gave a careers talk to students at Solent University, and through that I got to know a MSc student there who had previous GIS experience and was now doing a Data Analytics and AI MSc course. Her GIS experience was mostly in the ESRI stack (ArcGIS and related tools) and she was keen to learn other tools and how to combine her new Python and data knowledge with her previous GIS knowledge. I wrote her a long email with links to loads of resources and, with her permission, I’m publishing it here as it may be useful to others. The general focus is on the tools I use, which are mostly Python-focused, but also on becoming familiar with a range of tools rather than using tools from just one ecosystem (like ESRI). I hope it is useful to you.
Tools to investigate:
GDAL
GDAL is a library that consists of two parts GDAL and OGR. It provides ways to read and write geospatial data formats like shapefile, geopackage, GeoJSON, GeoTIFF etc – both raster (GDAL) and vector (OGR). It has a load of command-line tools like gdal_translate, ogr2ogr, gdalwarp and so on. These are extremely useful for converting between formats, importing data to databases, doing basic processing etc. It will be very useful for you to become familiar with the GDAL command-line tools. It comes with a Python interface which is a bit of a pain to use, and there are nicer libraries that are easier for using GDAL functionality from Python. A good tutorial for the command-line tools is at https://courses.spatialthoughts.com/gdal-tools.html
Command line tools in general
Getting familiar with running things from the command-line (Command Prompt on Windows) is very useful. On Windows I suggest installing ‘Windows Terminal’ (https://apps.microsoft.com/detail/9n0dx20hk701?hl=en-GB&gl=GB) and using that – but make sure you select standard command prompt not Powershell when you open a terminal using it.
Git
There’s a good tutorial at https://swcarpentry.github.io/git-novice/
qgis2web
This is a plugin for QGIS that will export a QGIS project to a standalone web map, with the styling/visualisation as close to the original QGIS project as they can manage. It’s very useful for exporting maps to share easily. There’s a tutorial at https://www.qgistutorials.com/en/docs/web_mapping_with_qgis2web.html and the main page is at https://plugins.qgis.org/plugins/qgis2web/
Python libraries to investigate:
explore() method, which will do an automatic webmap of a GeoPandas GeoDataFrame (like you did manually with Folium, but automatically)Cloud Native Geospatial
There’s a good ‘zine’ that explains the basics behind cloud-native geospatial – see https://zines.developmentseed.org/zines/cloud-native/. Understanding the basics of the topics in there would be good. There are loads of good tutorials online for using STAC catalogues, COG files and so on. See https://planetarycomputer.microsoft.com/docs/quickstarts/reading-stac/ and https://planetarycomputer.microsoft.com/docs/tutorials/cloudless-mosaic-sentinel2/ and https://github.com/microsoft/PlanetaryComputerExamples/blob/main/tutorials/surface_analysis.ipynb
My Blog
You can subscribe via email on the left-hand side at the bottom of the sidebar
Relevant posts:
Conference talks
These can be a really good way to get a brief intro to a topic, to know where to delve in deeper later. I often put them on and half-listen while I’m doing something else, and then switch to focusing on them fully if they get particularly interesting. There are loads of links here, don’t feel like you have to look at them all!
PostGIS Day conference:https://www.crunchydata.com/blog/postgis-day-2024-summary
Particularly relevant talks:
FOSS4G UK conference last year in Bristol:https://uk.osgeo.org/foss4guk2024/bristol.html
Most relevant talks for you are the following (just the slides):
FOSS4G conference YouTube videos:https://www.youtube.com/@FOSS4G/videos– they have a load of ones from 2022 at the top for some reason, but if you scroll down a long way you can find 2023 and 2024 stuff. Actually, better is to use this playlist of talks from the 2023 global conference: https://www.youtube.com/playlist?list=PLqa06jy1NEM2Kna9Gt_LDKZHv1dl4xUoZ
Here’s a few talks that might be particularly interesting/relevant to you, in no particular order
Suggestions for learning projects/tasks
(These are quite closely related to the MSc project that this student might be doing, but are probably useful for people generally)
I know when you’re starting off it is hard to work out what sort of things to do to develop your skills. One thing that is really useful is to become a bit of a ‘tool polyglot’, so you can do the same task in various tools depending on what makes sense in the circumstances.
I’ve listed a couple of tasks below. I’d suggest trying to complete them in a few ways:
Task 1 – Flood risk
Task 2 – Elevation data
(Don’t do this with PostGIS as its raster functionality isn’t great, but you could probably do all of this with GDAL command-line tools if you wanted)
Next, we'll look at some of the common user interface elements you've probably seen in many other applications — toolbars and menus. We'll also explore the neat system Qt provides for minimizing the duplication between different UI areas — QAction.
We'll start this tutorial with a simple skeleton application, which we can customize. Save the following code in a file named app.py -- this code all the imports you'll need for the later steps:
from PyQt6.QtCore import QSize, Qt
from PyQt6.QtGui import QAction, QIcon, QKeySequence
from PyQt6.QtWidgets import (
QApplication,
QCheckBox,
QLabel,
QMainWindow,
QStatusBar,
QToolBar,
)
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
app = QApplication([])
window = MainWindow()
window.show()
app.exec()
This file contains the imports and the basic code that you'll use to complete the examples in this tutorial.
If you're migrating to PyQt6 from PyQt5, notice that QAction is now available via the QtGui module.
One of the most commonly seen user interface elements is the toolbar. Toolbars are bars of icons and/or text used to perform common tasks within an application, for which access via a menu would be cumbersome. They are one of the most common UI features seen in many applications. While some complex applications, particularly in the Microsoft Office suite, have migrated to contextual 'ribbon' interfaces, the standard toolbar is usually sufficient for the majority of applications you will create.
Standard GUI elements
Let's start by adding a toolbar to our application.
In Qt, toolbars are created from the QToolBar class. To start, you create an instance of the class and then call addToolbar on the QMainWindow. Passing a string in as the first argument to QToolBar sets the toolbar's name, which will be used to identify the toolbar in the UI:
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
label = QLabel("Hello!")
label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.setCentralWidget(label)
toolbar = QToolBar("My main toolbar")
self.addToolBar(toolbar)
def onMyToolBarButtonClick(self, s):
print("click", s)
Run it! You'll see a thin grey bar at the top of the window. This is your toolbar. Right-click the name to trigger a context menu and toggle the bar off.
A window with a toolbar.
How can I get my toolbar back? Unfortunately, once you remove a toolbar, there is now no place to right-click to re-add it. So, as a general rule, you want to either keep one toolbar un-removeable, or provide an alternative interface in the menus to turn toolbars on and off.
We should make the toolbar a bit more interesting. We could just add a QButton widget, but there is a better approach in Qt that gets you some additional features — and that is via QAction. QAction is a class that provides a way to describe abstract user interfaces. What this means in English is that you can define multiple interface elements within a single object, unified by the effect that interacting with that element has.
For example, it is common to have functions that are represented in the toolbar but also the menu — think of something like Edit->Cut, which is present both in the Edit menu but also on the toolbar as a pair of scissors, and also through the keyboard shortcut Ctrl-X (Cmd-X on Mac).
Without QAction, you would have to define this in multiple places. But with QAction you can define a single QAction, defining the triggered action, and then add this action to both the menu and the toolbar. Each QAction has names, status messages, icons, and signals that you can connect to (and much more).
In the code below, you can see this first QAction added:
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
label = QLabel("Hello!")
label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.setCentralWidget(label)
toolbar = QToolBar("My main toolbar")
self.addToolBar(toolbar)
button_action = QAction("Your button", self)
button_action.setStatusTip("This is your button")
button_action.triggered.connect(self.onMyToolBarButtonClick)
toolbar.addAction(button_action)
def onMyToolBarButtonClick(self, s):
print("click", s)
To start with, we create the function that will accept the signal from the QAction so we can see if it is working. Next, we define the QAction itself. When creating the instance, we can pass a label for the action and/or an icon. You must also pass in any QObject to act as the parent for the action — here we're passing self as a reference to our main window. Strangely, for QAction the parent element is passed in as the final argument.
Next, we can opt to set a status tip — this text will be displayed on the status bar once we have one. Finally, we connect the triggered signal to the custom function. This signal will fire whenever the QAction is triggered (or activated).
Run it! You should see your button with the label that you have defined. Click on it, and then our custom method will print "click" and the status of the button.
Toolbar showing our QAction button.
Why is the signal always false? The signal passed indicates whether the button is checked, and since our button is not checkable — just clickable — it is always false. We'll show how to make it checkable shortly.
Next, we can add a status bar.
We create a status bar object by calling QStatusBar to get a new status bar object and then passing this into setStatusBar. Since we don't need to change the status bar settings, we can also just pass it in as we create it, in a single line:
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
label = QLabel("Hello!")
label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.setCentralWidget(label)
toolbar = QToolBar("My main toolbar")
self.addToolBar(toolbar)
button_action = QAction("Your button", self)
button_action.setStatusTip("This is your button")
button_action.triggered.connect(self.onMyToolBarButtonClick)
toolbar.addAction(button_action)
self.setStatusBar(QStatusBar(self))
def onMyToolBarButtonClick(self, s):
print("click", s)
Run it! Hover your mouse over the toolbar button, and you will see the status text in the status bar.
Status bar text updated as we hover over the action.
Next, we're going to turn our QAction toggleable — so clicking will turn it on, and clicking again will turn it off. To do this, we simply call setCheckable(True) on the QAction object:
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
label = QLabel("Hello!")
label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.setCentralWidget(label)
toolbar = QToolBar("My main toolbar")
self.addToolBar(toolbar)
button_action = QAction("Your button", self)
button_action.setStatusTip("This is your button")
button_action.triggered.connect(self.onMyToolBarButtonClick)
button_action.setCheckable(True)
toolbar.addAction(button_action)
self.setStatusBar(QStatusBar(self))
def onMyToolBarButtonClick(self, s):
print("click", s)
Run it! Click on the button to see it toggle from checked to unchecked state. Note that the custom slot method we create now alternates outputting True and False.
The toolbar button toggled on.
There is also a toggled signal, which only emits a signal when the button is toggled. But the effect is identical, so it is mostly pointless.
Things look pretty shabby right now — so let's add an icon to our button. For this, I recommend you download the fugue icon set by designer Yusuke Kamiyamane. It's a great set of beautiful 16x16 icons that can give your apps a nice professional look. It is freely available with only attribution required when you distribute your application — although I am sure the designer would appreciate some cash too if you have some spare.
Fugue Icon Set — Yusuke Kamiyamane
Select an image from the set (in the examples here, I've selected the file bug.png) and copy it into the same folder as your source code.
We can create a QIcon object by passing the file name to the class, e.g. QIcon("bug.png") -- if you place the file in another folder, you will need a full relative or absolute path to it.
Finally, to add the icon to the QAction (and therefore the button), we simply pass it in as the first argument when creating the QAction.
You also need to let the toolbar know how large your icons are. Otherwise, your icon will be surrounded by a lot of padding. You can do this by calling setIconSize() with a QSize object:
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
label = QLabel("Hello!")
label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.setCentralWidget(label)
toolbar = QToolBar("My main toolbar")
toolbar.setIconSize(QSize(16, 16))
self.addToolBar(toolbar)
button_action = QAction(QIcon("bug.png"), "Your button", self)
button_action.setStatusTip("This is your button")
button_action.triggered.connect(self.onMyToolBarButtonClick)
button_action.setCheckable(True)
toolbar.addAction(button_action)
self.setStatusBar(QStatusBar(self))
def onMyToolBarButtonClick(self, s):
print("click", s)
Run it! The QAction is now represented by an icon. Everything should work exactly as it did before.
Our action button with an icon.
Note that Qt uses your operating system's default settings to determine whether to show an icon, text, or an icon and text in the toolbar. But you can override this by using setToolButtonStyle(). This slot accepts any of the following flags from the Qt namespace:
| Flag | Behavior |
|---|---|
Qt.ToolButtonStyle.ToolButtonIconOnly | Icon only, no text |
Qt.ToolButtonStyle.ToolButtonTextOnly | Text only, no icon |
Qt.ToolButtonStyle.ToolButtonTextBesideIcon | Icon and text, with text beside the icon |
Qt.ToolButtonStyle.ToolButtonTextUnderIcon | Icon and text, with text under the icon |
Qt.ToolButtonStyle.ToolButtonFollowStyle | Follow the host desktop style |
The default value is Qt.ToolButtonStyle.ToolButtonFollowStyle, meaning that your application will default to following the standard/global setting for the desktop on which the application runs. This is generally recommended to make your application feel as native as possible.
Finally, we can add a few more bits and bobs to the toolbar. We'll add a second button and a checkbox widget. As mentioned, you can literally put any widget in here, so feel free to go crazy:
from PyQt6.QtCore import QSize, Qt
from PyQt6.QtGui import QAction, QIcon
from PyQt6.QtWidgets import (
QApplication,
QCheckBox,
QLabel,
QMainWindow,
QStatusBar,
QToolBar,
)
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
label = QLabel("Hello!")
label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.setCentralWidget(label)
toolbar = QToolBar("My main toolbar")
toolbar.setIconSize(QSize(16, 16))
self.addToolBar(toolbar)
button_action = QAction(QIcon("bug.png"), "&Your button", self)
button_action.setStatusTip("This is your button")
button_action.triggered.connect(self.onMyToolBarButtonClick)
button_action.setCheckable(True)
toolbar.addAction(button_action)
toolbar.addSeparator()
button_action2 = QAction(QIcon("bug.png"), "Your &button2", self)
button_action2.setStatusTip("This is your button2")
button_action2.triggered.connect(self.onMyToolBarButtonClick)
button_action2.setCheckable(True)
toolbar.addAction(button_action2)
toolbar.addWidget(QLabel("Hello"))
toolbar.addWidget(QCheckBox())
self.setStatusBar(QStatusBar(self))
def onMyToolBarButtonClick(self, s):
print("click", s)
app = QApplication([])
window = MainWindow()
window.show()
app.exec()
Run it! Now you see multiple buttons and a checkbox.
Toolbar with an action and two widgets.
Menus are another standard component of UIs. Typically, they are at the top of the window or the top of a screen on macOS. They allow you to access all standard application functions. A few standard menus exist — for example File, Edit, Help. Menus can be nested to create hierarchical trees of functions, and they often support and display keyboard shortcuts for fast access to their functions.
Standard GUI elements - Menus
To create a menu, we create a menubar we call menuBar() on the QMainWindow. We add a menu to our menu bar by calling addMenu(), passing in the name of the menu. I've called it '&File'. The ampersand defines a quick key to jump to this menu when pressing Alt.
This won't be visible on macOS. Note that this is different from a keyboard shortcut — we'll cover that shortly.
This is where the power of actions comes into play. We can reuse the already existing QAction to add the same function to the menu. To add an action, you call addAction() passing in one of our defined actions:
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
label = QLabel("Hello!")
label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.setCentralWidget(label)
toolbar = QToolBar("My main toolbar")
toolbar.setIconSize(QSize(16, 16))
self.addToolBar(toolbar)
button_action = QAction(QIcon("bug.png"), "&Your button", self)
button_action.setStatusTip("This is your button")
button_action.triggered.connect(self.onMyToolBarButtonClick)
button_action.setCheckable(True)
toolbar.addAction(button_action)
toolbar.addSeparator()
button_action2 = QAction(QIcon("bug.png"), "Your &button2", self)
button_action2.setStatusTip("This is your button2")
button_action2.triggered.connect(self.onMyToolBarButtonClick)
button_action2.setCheckable(True)
toolbar.addAction(button_action2)
toolbar.addWidget(QLabel("Hello"))
toolbar.addWidget(QCheckBox())
self.setStatusBar(QStatusBar(self))
menu = self.menuBar()
file_menu = menu.addMenu("&File")
file_menu.addAction(button_action)
def onMyToolBarButtonClick(self, s):
print("click", s)
Run it! Click the item in the menu, and you will notice that it is toggleable — it inherits the features of the QAction.
Menu shown on the window -- on macOS this will be at the top of the screen.
Let's add some more things to the menu. Here, we'll add a separator to the menu, which will appear as a horizontal line in the menu, and then add the second QAction we created:
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
label = QLabel("Hello!")
label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.setCentralWidget(label)
toolbar = QToolBar("My main toolbar")
toolbar.setIconSize(QSize(16, 16))
self.addToolBar(toolbar)
button_action = QAction(QIcon("bug.png"), "&Your button", self)
button_action.setStatusTip("This is your button")
button_action.triggered.connect(self.onMyToolBarButtonClick)
button_action.setCheckable(True)
toolbar.addAction(button_action)
toolbar.addSeparator()
button_action2 = QAction(QIcon("bug.png"), "Your &button2", self)
button_action2.setStatusTip("This is your button2")
button_action2.triggered.connect(self.onMyToolBarButtonClick)
button_action2.setCheckable(True)
toolbar.addAction(button_action2)
toolbar.addWidget(QLabel("Hello"))
toolbar.addWidget(QCheckBox())
self.setStatusBar(QStatusBar(self))
menu = self.menuBar()
file_menu = menu.addMenu("&File")
file_menu.addAction(button_action)
file_menu.addSeparator()
file_menu.addAction(button_action2)
def onMyToolBarButtonClick(self, s):
print("click", s)
Run it! You should see two menu items with a line between them.
Our actions showing in the menu.
You can also use ampersand to add accelerator keys to the menu to allow a single key to be used to jump to a menu item when it is open. Again this doesn't work on macOS.
To add a submenu, you simply create a new menu by calling addMenu() on the parent menu. You can then add actions to it as usual. For example:
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
label = QLabel("Hello!")
label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.setCentralWidget(label)
toolbar = QToolBar("My main toolbar")
toolbar.setIconSize(QSize(16, 16))
self.addToolBar(toolbar)
button_action = QAction(QIcon("bug.png"), "&Your button", self)
button_action.setStatusTip("This is your button")
button_action.triggered.connect(self.onMyToolBarButtonClick)
button_action.setCheckable(True)
toolbar.addAction(button_action)
toolbar.addSeparator()
button_action2 = QAction(QIcon("bug.png"), "Your &button2", self)
button_action2.setStatusTip("This is your button2")
button_action2.triggered.connect(self.onMyToolBarButtonClick)
button_action2.setCheckable(True)
toolbar.addAction(button_action2)
toolbar.addWidget(QLabel("Hello"))
toolbar.addWidget(QCheckBox())
self.setStatusBar(QStatusBar(self))
menu = self.menuBar()
file_menu = menu.addMenu("&File")
file_menu.addAction(button_action)
file_menu.addSeparator()
file_submenu = file_menu.addMenu("Submenu")
file_submenu.addAction(button_action2)
def onMyToolBarButtonClick(self, s):
print("click", s)
Run it! You will see a nested menu in the File menu.
Submenu nested in the File menu.
Finally, we'll add a keyboard shortcut to the QAction. You define a keyboard shortcut by passing setKeySequence() and passing in the key sequence. Any defined key sequences will appear in the menu.
Note that the keyboard shortcut is associated with the QAction and will still work whether or not the QAction is added to a menu or a toolbar.
Key sequences can be defined in multiple ways - either by passing as text, using key names from the Qt namespace, or using the defined key sequences from the Qt namespace. Use the latter wherever you can to ensure compliance with the operating system standards.
The completed code, showing the toolbar buttons and menus, is shown below:
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("My App")
label = QLabel("Hello!")
# The `Qt` namespace has a lot of attributes to customize
# widgets. See: http://doc.qt.io/qt-6/qt.html
label.setAlignment(Qt.AlignmentFlag.AlignCenter)
# Set the central widget of the Window. Widget will expand
# to take up all the space in the window by default.
self.setCentralWidget(label)
toolbar = QToolBar("My main toolbar")
toolbar.setIconSize(QSize(16, 16))
self.addToolBar(toolbar)
button_action = QAction(QIcon("bug.png"), "&Your button", self)
button_action.setStatusTip("This is your button")
button_action.triggered.connect(self.onMyToolBarButtonClick)
button_action.setCheckable(True)
# You can enter keyboard shortcuts using key names (e.g. Ctrl+p)
# Qt.namespace identifiers (e.g. Qt.CTRL + Qt.Key_P)
# or system agnostic identifiers (e.g. QKeySequence.Print)
button_action.setShortcut(QKeySequence("Ctrl+p"))
toolbar.addAction(button_action)
toolbar.addSeparator()
button_action2 = QAction(QIcon("bug.png"), "Your &button2", self)
button_action2.setStatusTip("This is your button2")
button_action2.triggered.connect(self.onMyToolBarButtonClick)
button_action2.setCheckable(True)
toolbar.addAction(button_action2)
toolbar.addWidget(QLabel("Hello"))
toolbar.addWidget(QCheckBox())
self.setStatusBar(QStatusBar(self))
menu = self.menuBar()
file_menu = menu.addMenu("&File")
file_menu.addAction(button_action)
file_menu.addSeparator()
file_submenu = file_menu.addMenu("Submenu")
file_submenu.addAction(button_action2)
def onMyToolBarButtonClick(self, s):
print("click", s)
Experiment with building your own menus using QAction and QMenu.
When sorting strings, you’d often like the order to make sense to a person. That means numbers need to be treated numerically even if they are in a larger string.
For example, sorting Python versions with the default sort() would give you:
Python 3.10
Python 3.11
Python 3.9
when you want it to be:
Python 3.9
Python 3.10
Python 3.11
I wrote about this long ago (Human sorting), but have continued to tweak the code and needed to add it to a project recently. Here’s the latest:
import re
def human_key(s: str) -> tuple[list[str | int], str]:
"""Turn a string into a sortable value that works how humans expect.
"z23A" -> (["z", 23, "a"], "z23A")
The original string is appended as a last value to ensure the
key is unique enough so that "x1y" and "x001y" can be distinguished.
"""
def try_int(s: str) -> str | int:
"""If `s` is a number, return an int, else `s` unchanged."""
try:
return int(s)
except ValueError:
return s
return ([try_int(c) for c in re.split(r"(\d+)", s.casefold())], s)
def human_sort(strings: list[str]) -> None:
"""Sort a list of strings how humans expect."""
strings.sort(key=human_key)
The central idea here is to turn a string like "Python 3.9" into the
key ["Python ", 3, ".", 9] so that numeric components will be sorted by
their numeric value. The re.split() function gives us interleaved words and
numbers, and try_int() turns the numbers into actual numbers, giving us sortable
key lists.
There are two improvements from the original:
"x1y" and
"x001Y" would both produce the same key. This solves a
problem that actually happened when sorting the items of
a dictionary.
# Without the tuple: different strings, same key!!
human_key("x1y") -> ["x", 1, "y"]
human_key("x001Y") -> ["x", 1, "y"]
# With the tuple: different strings, different keys.
human_key("x1y") -> (["x", 1, "y"], "x1y")
human_key("x001Y") -> (["x", 1, "y"], "x001Y")
If you are interested, there are many different ways to split the string into the word/number mix. The comments on the old post have many alternatives, and there are certainly more.
This still makes some assumptions about what is wanted, and doesn’t cover all possible options (floats? negative/positive? full file paths?). For those, you probably want the full-featured natsort (natural sort) package.
Every ending marks a new beginning, and today, is the beginning of a new chapter for me. Ten years ago I took a leap into the unknown, today I take another. After a decade of working on Sentry I move on to start something new.
Sentry has been more than just a job, it has been a defining part of my life. A place where I've poured my energy, my ideas, my heart. It has shaped me, just as I've shaped it. And now, as I step away, I do so with immense gratitude, a deep sense of pride, and a heart full of memories.
I've known David, Sentry's co-founder (alongside Chris), long before I was ever officially part of the team as our paths first crossed on IRC in the Django community. Even my first commit to Sentry predates me officially working there by a few years. Back in 2013, over conversations in the middle of Russia — at a conference that, incidentally, also led to me meeting my wife — we toyed with the idea of starting a company together. That exact plan didn't materialize, but the seeds of collaboration had been planted.
Conversations continued, and by late 2014, the opportunity to help transform Sentry (which already showed product market fit) into a much bigger company was simply too good to pass up. I never could have imagined just how much that decision would shape the next decade of my life.
For me, Sentry's growth has been nothing short of extraordinary. At first, I thought reaching 30 employees would be our ceiling. Then we surpassed that, and the milestones just kept coming — reaching a unicorn valuation was something I once thought was impossible. While we may have stumbled at times, we've also learned immensely throughout this time.
I'm grateful for all the things I got to experience and there never was a dull moment. From representing Sentry at conferences, opening an engineering office in Vienna, growing teams, helping employees, assisting our licensing efforts and leading our internal platform teams. Every step and achievement drove me.
Yet for me, the excitement and satisfaction of being so close to the founding of a company, yet not quite a founder, has only intensified my desire to see the rest of it.
Walking away from something you love is never easy and leaving Sentry is hard. Really hard. Sentry has been woven into the very fabric of my adult life. Working on it hasn't just spanned any random decade; it perfectly overlapped with marrying my wonderful wife, and growing our family from zero to three kids.
And will it go away entirely? The office is right around the corner afterall. From now on, every morning, when I will grab my coffee, I will walk past it. The idea of no longer being part of the daily decisions, the debates, the momentum — it feels surreal. That sense of belonging to a passionate team, wrestling with tough decisions, chasing big wins, fighting fires together, sometimes venting about our missteps and discussing absurd and ridiculous trivia became part of my identity.
There are so many bright individuals at Sentry, and I'm incredibly proud of what we have built together. Not just from an engineering point of view, but also product, marketing and upholding our core values. We developed SDKs that support a wide array of platforms from Python to JavaScript to Swift to C++, lately expanding to game consoles. We stayed true to our Open Source principles, even when other options were available. For example, when we needed an Open Source PDB implementation for analyzing Windows crashes but couldn't find a suitable solution, we contributed to a promising Rust crate instead of relying on Windows VMs and Microsoft's dbghelp. When we started, our ingestion system handled a few thousand requests per second — now it handles well over a million.
While building an SDK may seem straightforward, maintaining and updating them to remain best-in-class over the years requires immense dedication. It takes determination to build something that works out of the box with little configuration. A lot of clever engineering and a lot of deliberate tradeoffs went into the product to arrive where it is. And ten years later, is a multi-product company. What started with just crashes, now you can send traces, profiles, sessions, replays and more.
We also stuck to our values. I'm pleased that we ran experiments with licensing despite all the push back we got over the years. We might not have found the right solution yet, but we pushed the conversation. The same goes for our commitment to funding of dependencies.
I feel an enormous amount of gratitude for those last ten years. There are so many people I owe thanks to. I owe eternal thanks to David Cramer and Chris Jennings for the opportunity and trust they placed in me. To Ben Vinegar for his unwavering guidance and support. To Dan Levine, for investing in us and believing in our vision. To Daniel Griesser, for being an exceptional first hire in Vienna, and shepherding our office there and growing it to 50 people. To Vlad Cretu, for bringing structure to our chaos over the years. To Milin Desai for taking the helm and growing us.
And most of all, to my wonderful wife, Maria — who has stood beside me through every challenge, who has supported me when the road was uncertain, and who has always encouraged me to forge my own path.
To everyone at Sentry, past and present — thank you. For the trust, the lessons, the late nights, the victories. For making Sentry what it is today.
I'm fully aware it's a gamble to believe my next venture will find the same success as Sentry. The reality is that startups that achieve the kind of scale and impact Sentry has are incredibly rare. There's a measure of hubris in assuming lightning strikes twice, and as humbling as that realization is, it also makes me that much more determined. The creative spark that fueled me at Sentry isn't dimming. Not at all in fact: it burns brighter fueld by the feeling that I can explore new things, beckoning me. There's more for me to explore, and I'm ready to channel all that energy into a new venture.
Today, I stand in an open field, my backpack filled with experiences and a renewed sense of purpose. That's because the world has changed a lot in the past decade, and so have I. What drives me now is different from what drove me before, and I want my work to reflect that evolution.
At my core, I'm still inspired by the same passion — seeing others find value in what I create, but my perspective has expanded. While I still take great joy in building things that help developers, I want to broaden my reach. I may not stray far from familiar territory, but I want to build something that speaks to more people, something that, hopefully, even my children will find meaningful.
Watch this space, as they say.
I have for as long as I can remember had a bit of a problem with analysis paralysis and tunnel vision.
If I’m working on a problem and get stuck, I have a tendency to just sit there paging through code trying to understand where to go next. It’s a very unproductive habit and one I’m committed to breaking, because the last thing you want is to lose hours of wall clock time with no progress on your work.
I was talking to my boss about this a few weeks back when I had a crazy idea: “Hey what if I wrote a program that looked for a particular key combo that I’d hit every time I make progress, and if a specified period e.g. 15 or 30 minutes go by with no progress, a loud buzzer gets played to remind me to ask for help, take a break, or just try something different.”
He thought this was a great idea, and suggested that this would be an ideal candidate to try as an “AI speed run”.
This article is a brief exploration of the process I used with some concrete hints on things that helped me make this project a success that you can use in your own coding “speed run” endeavors 
For purposes of this discussion I used ChatGPT with its GPT4.0 model. There’s nothing magical about that choice, you can use Claude or any other LLM that fits your needs.
Now comes the important part – coming up with the prompt! The first and most important part of building any program is coming to a full and detailed understanding of what you want to build.
Be as descriptive as you can, being sure to include all the most salient aspects of your project.
What does it do? Here’s where detail and specifics are super important. Where does it need to run? In a web browser? Windows? Mac? Linux? These are just examples of the kinds of detail you must include.
The initial prompt I came up with was: “Write a program that will run on Mac, Windows and Linux. The program should listen for a particular key combination, and if it doesn’t receive that combination within a prescribed (configurable) time, it plays a notification sound to the user.”.
Building software with a large language model isn’t like rubbing a magic lamp and making a wish, asking for your software to appear.
Instead, it’s more like having a conversation about what you want to build with an artist about something you want them to create for you.
The LLM is almost guaranteed to not produce exactly what you want on the first try. You can find the complete transcript of my conversation with ChatGPT for this project here.
Do take a moment to read through it a bit. Notice that on the first try it didn’t work at all, so I told it that and gave it the exact error. The fix it suggested wasn’t helping, so I did a tiny bit of very basic debugging and found that one of the modules it was suggested (the one for keyboard input) blew up as soon as I ran its import. So I told it that and suggested that the problem was with the other module that played the buzzer sound.
Once we got past all the platform specific library shenanigans, there were structural issues with the code that needed to be addressed. When I ran the code it generated I got this:
UnboundLocalError: cannot access local variable 'watchdog_last_activity' where it is not associated with a valueSo I told it that by feeding the error back in. It then corrected course and generated the first fully working version of the program. Success!
And I don’t know about you, but a detail about this process that still amazes me? This whole conversation took less than an hour from idea to working program! That’s quite something.
When Bob suggested that I should publish my project to the Python package repository I loved the idea, but I’d never done this before. Lately I’ve been using the amazing uv for all things package related. It’s an amazing tool!
So I dug into the documentation and started playing with my pyproject.toml. And if I’m honest? It wasn’t going very well. I kept trying to run uv publish and kept getting what seemed to me like inscrutable metadata errors
At moments like that I try to ask myself one simple question: “Am I following the happy path?” and in this case, the answer was no
When I started this project, I had used the uv init command to set up the project. I began to wonder whether I had set things up wrong, so I pored over the uv docs and one invocation of uv init --package later I had a buildable package that I could publish to pypi!
There was one bit of polish remaining before I felt like I could call this project “done” as a minimum viable product.
One of the things I’d struggled with since I first tried to package the program was where to put and how to bundle the sound file for the buzzer.
After trying various unsatisfying and sub-optimal things like asking the user to supply their own and using a command line argument to locate it, one of Bob’s early suggestions came to mind: I really needed to bundle the sound inside the package in such a way that the program could load it at run time.
One of the things you learn as you start working with large language models is that they act like a really good pair programming buddy. They offer another place to turn when you get stuck. So I asked ChatGPT:
Write a pyproject.toml for a Python package that includes code that loads a sound file from inside the package.
That did the trick! ChatGPT gave me the right pointers to include in my project toml file as well as the Python code to load the included sound file at run time!
As you can see from the final code, this program uses cross platform Python modules for sound playback and keyboard input and more importantly uses threads to manage the real time capture of keypresses while keeping track of the time.
I’ve been in this industry for over 30 years, and a recurring theme I’ve been hearing for most of that time is “Threads are hard”. And they are! But there are also cases like this where you can use them simply and reliably where they really make good sense! I know that now, and would feel comforable using them this way in a future project. There’s value in that! Any tool we can use to help us grow and improve our skills is one worth using, and if we take the time to understand the code AI generates for us it’s a good investment in my book!
I’m very grateful to my manager for having suggested that I try building this project as an “AI speed run”. It’s not something that would have occurred to me but in the end analysis it was a great experience from which I learned a lot.
Also? I’m super happy with the resulting tool and use it all the time now to ensure I don’t stay stuck and burn a ton of time making no progress!
You can see the project in its current state on my GitHub. There are lots of ideas I have for extending it in the future including a nice Textual interface and more polish around choosing the key chord and the “buzzer” sound.
Thanks for taking the time to read this. I hope that it inspires you to try your own AI speed run!