Introduction
If you're majoring in Computer Science, Insertion Sort is most likely one of the first sorting algorithms you have heard of. It is intuitive and easy to implement, but it's very slow on large arrays and is almost never used to sort them.
Insertion sort is often illustrated by comparing it to sorting a hand of cards while playing rummy. For those of you unfamiliar with the game, most players want the cards in their hand sorted in ascending order so they can quickly see which combinations they have at their disposal.
The dealer deals out 14 cards to you, you pick them up one by one and put them in your hand in the right order. During this entire process your hand holds the sorted "subarray" of your cards, while the remaining face down cards on the table are unsorted - from which you take cards one by one, and put them in your hand.
Insertion Sort is very simple and intuitive to implement, which is one of the reasons it's generally taught at an early stage in programming. It's a stable, in-place algorithm, that works really well for nearly-sorted or small arrays.
Let's elaborate these terms:
- in-place: Requires a small, constant additional space (no matter the input size of the collection), but rewrites the original memory locations of the elements in the collection.
- stable: The algorithm maintains the relative order of equal objects from the initial array. In other words, say your company's employee database returns "Dave Watson" and "Dave Brown" as two employees, in that specific order. If you were to sort them by their (shared) first name, a stable algorithm would guarantee that this order remains unchanged.
Another thing to note: Insertion Sort doesn't need to know the entire array in advance before sorting. The algorithm can receive one element at a time. Which is great if we want to add more elements to be sorted - the algorithm only inserts that element in it's proper place without "re-doing" the whole sort.
Insertion Sort is used rather often in practice, because of how efficient it is for small (~10 item) data sets. We will talk more about that later.
How Insertion Sort Works
An array is partitioned into a "sorted" subarray and an "unsorted" subarray. At the beginning, the sorted subarray contains only the first element of our original array.
The first element in the unsorted array is evaluated so that we can insert it into its proper place in the sorted subarray.
The insertion is done by moving all elements larger than the new element one position to the right.
Continue doing this until our entire array is sorted.
Keep in mind however that when we say an element is larger or smaller than another element - it doesn't necessarily mean larger or smaller integers.
We can define the words "larger" and "smaller" however we like when using custom objects. For example, point A can be "larger" than point B if it's further away from the center of the coordinate system.
We will mark the sorted subarray with bolded numbers, and use the following array to illustrate the algorithm:
8, 5, 4, 10, 9
The first step would be to "add" 8 to our sorted subarray.
8, 5, 4, 10, 9
Now we take a look at the first unsorted element - 5. We keep that value in a separate variable, for example current, for safe-keeping. 5 is less than 8. We move 8 one place to the right, effectively overwriting the 5 that was previously stored there (hence the separate variable for safe keeping):
8, 8, 4, 10, 9 (current = 5)
5 is lesser than all the elements in our sorted subarray, so we insert it in to the first position:
5, 8, 4, 10, 9
Next we look at number 4. We save that value in current. 4 is less than 8 so we move 8 to the right, and do the same with 5.
5, 5, 8, 10, 9 (current = 4)
Again we've encountered an element lesser than our entire sorted subarray, so we put it in the first position:
4, 5, 8, 10, 9
10 is greater than our rightmost element in the sorted subarray and is therefore larger than any of the elements to the left of 8. So we simply move on to the next element:
4, 5, 8, 10, 9
9 is less than 10, so we move 10 to the right:
4, 5, 8, 10, 10 (current = 9)
However, 9 is greater than 8, so we simply insert 9 right after 8.
4, 5, 8, 9, 10
Implementation
As we have previously mentioned, Insertion Sort is fairly easy to implement. We'll implement it first on a simple array of integers and then on some custom objects.
In practice, it's much more likely that you'll be working with objects and sorting them based on certain criteria.
Sorting Arrays
def insertion_sort(array):
# We start from 1 since the first element is trivially sorted
for index in range(1, len(array)):
currentValue = array[index]
currentPosition = index
# As long as we haven't reached the beginning and there is an element
# in our sorted array larger than the one we're trying to insert - move
# that element to the right
while currentPosition > 0 and array[currentPosition - 1] > currentValue:
array[currentPosition] = array[currentPosition -1]
currentPosition = currentPosition - 1
# We have either reached the beginning of the array or we have found
# an element of the sorted array that is smaller than the element
# we're trying to insert at index currentPosition - 1.
# Either way - we insert the element at currentPosition
array[currentPosition] = currentValue
Let's populate a simple array and sort it:
array = [4, 22, 41, 40, 27, 30, 36, 16, 42, 37, 14, 39, 3, 6, 34, 9, 21, 2, 29, 47]
insertion_sort(array)
print("sorted array: " + str(array))
Output:
sorted array: [2, 3, 4, 6, 9, 14, 16, 21, 22, 27, 29, 30, 34, 36, 37, 39, 40, 41, 42, 47]
Note: It would have been technically incorrect if we reversed the order of conditions in our while loop. That is, if we first checked whether array[currentPosition-1] > currentValue before checking whether currentPosition > 0.
This would mean that if we did indeed reach the 0th element, we would first check whether array[-1] > currentValue. In Python this is "fine", although technically incorrect, since it wouldn't cause the loop to end prematurely or continue when it shouldn't.
This is because if we had reached the zeroth element, the second condition of currentPosition > 0 would fail regardless of the first condition, and cause the loop to break. In Python array[-1] > currentValue is equivalent to array[len(array) - 2] > currentValue and the interpreter wouldn't complain, but this is not a comparison we actually want to happen.
This reversed order of conditions is a bad idea. In a lot of cases it can lead to unexpected results that can be hard to debug because there is no syntactic or semantic error. Most programming languages would "complain" for trying to access the -1st element, but Python won't complain and it's easy to miss such a mistake.
The takeaway advice from this is to always check whether the index is valid before using it to access elements..
Sorting Custom Objects
We've mentioned before that we can define "greater than" and "lesser than" however we want - that the definition didn't need to rely solely on integers. There are several ways we can change our algorithm to work with custom objects.
We could redefine the actual comparison operators for our custom class and keep the same algorithm code as above. However, that would mean that we'd need to overload those operators if we wanted to sort the objects of our class in a different way.
Arguably the best way to use Insertion Sort for custom classes is to pass another argument to the insertion_sort method - specifically a comparison method. The most convenient way to do this is by using a custom lambda function when calling the sorting method.
In this implementation, we'll sort points where a "smaller" point is the one with a lower x coordinate.
First we'll define our Point class:
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __str__(self):
return str.format("({},{})", self.x, self.y)
Then we make a slight change to our insertion_sort method to accommodate custom sorting:
def insertion_sort(array, compare_function):
for index in range(1, len(array)):
currentValue = array[index]
currentPosition = index
while currentPosition > 0 and compare_function(array[currentPosition - 1], currentValue):
array[currentPosition] = array[currentPosition - 1]
currentPosition = currentPosition - 1
array[currentPosition] = currentValue
Finally we test the program:
A = Point(1,2)
B = Point(4,4)
C = Point(3,1)
D = Point(10,0)
array = [A,B,C,D]
# We sort by the x coordinate, ascending
insertion_sort(array, lambda a, b: a.x > b.x)
for point in array:
print(point)
We get the output:
(1,2)
(3,1)
(4,4)
(10,0)
This algorithm will now work for any type of array, as long as we provide an appropriate comparison function.
Insertion Sort in Practice
Insertion sort may seem like a slow algorithm, and indeed in most cases it is too slow for any practical use with its O(n2) time complexity. However, as we've mentioned, it's very efficient on small arrays and on nearly sorted arrays.
This makes Insertion Sort very relevant for use in combination with algorithms that work well on large data sets.
For example, Java used a Dual Pivot Quick Sort as the primary sorting algorithm, but used Insertion Sort whenever the array (or subarray created by Quick Sort) had less than 7 elements.
Another efficient combination is simply ignoring all small subarrays created by Quick Sort, and then passing the final, nearly-sorted array, through Insertion Sort.
Another place where Insertion Sort left its mark is with a very popular algorithm called Shell Sort. Shell Sort works by calling Insertion Sort to sort pairs of elements far apart from each other, then incrementally reducing the gap between elements to should be compared.
Essentially making a lot of Insertion Sort calls to first small, and then nearly-sorted larger arrays, harnessing all the advantages it can.
Conclusion
Insertion Sort is a very simple, generally inefficient algorithm that nonetheless has several specific advantages that make it relevant even after many other, generally more efficient algorithms have been developed.
It remains a great algorithm for introducing future software developers into the world of sorting algorithms, and is still used in practice for specific scenarios in which it shines.
