tl;dr
On Thursday 5 September 2024 we performed some system maintenance. It appeared to have gone well, and was completed at the scheduled time (06:20 UTC), but unfortunately there were unexpected knock-on effects that caused issues later on in the day, and further problems on Saturday 7 September. This post gives the details of why we needed to perform the maintenance, what happened, and what we will do to prevent a recurrence.