Pandas dataframe is a great tool for handling tabular data in python. In this article, we will discuss different ways to check if a column is sorted in a pandas dataframe.
Check if a Column Is Sorted Using Column Attributes
To check if a column is sorted either in ascending order in a pandas dataframe, we can use the is_monotonic attribute of the column. The is_monotonic attribute evaluates to True if a column is sorted in ascending order i.e. if values in the column are monotonically increasing.
For instance, if a dataframe is sorted in ascending order, the is_monotonic attribute will evaluate to True as shown below.
import pandas as pd
df=pd.read_csv("grade2.csv")
df.sort_values(by="Marks",inplace=True)
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
7 3 11 Bobby 50 D
0 2 27 Harsh 55 C
5 3 27 Aditya 55 C
1 2 23 Clara 78 B
4 3 15 Prashant 78 B
6 3 23 Radheshyam 78 B
2 3 33 Tina 82 A
3 3 34 Amy 88 A
The 'Marks' column is sorted: TrueIn the above example, we first loaded a CSV file into a dataframe using the read_csv() function. After that, we sorted the dataframe by the "Marks" column using the sort_values() method. After sorting, you can observe that the is_monotonic attribute of the column returns True. It denotes that the column is sorted in descending order.
If a column is sorted in descending order, the is_monotonic attribute will evaluate to False.
import pandas as pd
df=pd.read_csv("grade2.csv")
df.sort_values(by="Marks",inplace=True,ascending=False)
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
3 3 34 Amy 88 A
2 3 33 Tina 82 A
1 2 23 Clara 78 B
4 3 15 Prashant 78 B
6 3 23 Radheshyam 78 B
0 2 27 Harsh 55 C
5 3 27 Aditya 55 C
7 3 11 Bobby 50 D
The 'Marks' column is sorted: FalseIn this example, we have sorted the "Marks" column in descending order. Due to this, the is_monotonic attribute evaluates to False.
If a column in the dataframe is not sorted, the is_monotonic attribute will evaluate to False. You can observe this in the following example.
import pandas as pd
df=pd.read_csv("grade2.csv")
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
0 2 27 Harsh 55 C
1 2 23 Clara 78 B
2 3 33 Tina 82 A
3 3 34 Amy 88 A
4 3 15 Prashant 78 B
5 3 27 Aditya 55 C
6 3 23 Radheshyam 78 B
7 3 11 Bobby 50 D
The 'Marks' column is sorted: FalseHere, you can observe that we have accessed the is_monotonic attribute without sorting the dataframe by the "Marks"column. Hence, the "Marks"column is unsorted and the is_monotonic attribute evaluates to False.
The is_monotonic doesn’t work with NaN values. If a column contains NaN values, the is_monotonic attribute always evaluates to False. You can observe this in the following example.
import pandas as pd
df=pd.read_csv("grade.csv")
df.sort_values(by="Marks",inplace=True,ascending=True)
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
6 2 27 Harsh 55.0 C
10 3 27 Aditya 55.0 C
4 2 22 Tom 73.0 B
2 1 14 Sam 75.0 B
5 2 15 Golu 79.0 B
0 1 11 Aditya 85.0 A
8 3 34 Amy 88.0 A
1 1 12 Chris NaN A
3 1 15 Harry NaN NaN
7 2 23 Clara NaN B
9 3 15 Prashant NaN B
11 3 23 Radheshyam NaN NaN
The 'Marks' column is sorted: FalseIn this example, you can observe that the "Marks" column contains NaN values. Due to this, even after sorting, the is_monotonic attribute evaluates to False. You may argue that the NaN values are at the last of the column. Maybe, this is why the is_monotonic attribute evaluates to False.
However, if we put the rows having NaN values at the top of the dataframe, the is_monotonic attribute will again evaluate to False. You can observe this in the following example.
import pandas as pd
df=pd.read_csv("grade.csv")
df.sort_values(by="Marks",inplace=True,ascending=True,na_position="first")
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
1 1 12 Chris NaN A
3 1 15 Harry NaN NaN
7 2 23 Clara NaN B
9 3 15 Prashant NaN B
11 3 23 Radheshyam NaN NaN
6 2 27 Harsh 55.0 C
10 3 27 Aditya 55.0 C
4 2 22 Tom 73.0 B
2 1 14 Sam 75.0 B
5 2 15 Golu 79.0 B
0 1 11 Aditya 85.0 A
8 3 34 Amy 88.0 A
The 'Marks' column is sorted: FalseIn this example, we have put the NaN values at the start of the sorted "Marks" column. Even after this, the is_monotonic attribute evaluates to False. Thus, we can conclude that the is_monotonic attribute cannot be used with columns having NaN values.
While using the is_monotonic attribute, you will get a FutureWarning with the message “FutureWarning: is_monotonic is deprecated and will be removed in a future version. Use is_monotonic_increasing instead.” So, the is_monotonic attribute will be deprecated in future pandas versions. As an alternative, we can use the is_monotonic_increasing and is_monotonic_decreasing attributes to check if a column is sorted in a pandas dataframe.
Check if a Column Is Sorted in Ascending Order in a Dataframe
To check if a column in a dataframe is sorted in ascending order, we can use the is_monotonic_increasing attribute. The is_monotonic_increasing attribute evaluates to True if a column is sorted in ascending order. Otherwise, it is set to False. You can observe this in the following example.
import pandas as pd
df=pd.read_csv("grade2.csv")
df.sort_values(by="Marks",inplace=True,ascending=True)
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic_increasing
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
7 3 11 Bobby 50 D
0 2 27 Harsh 55 C
5 3 27 Aditya 55 C
1 2 23 Clara 78 B
4 3 15 Prashant 78 B
6 3 23 Radheshyam 78 B
2 3 33 Tina 82 A
3 3 34 Amy 88 A
The 'Marks' column is sorted: TrueIf a column is not sorted, the is_monotonic_increasing attribute evaluates to False.
import pandas as pd
df=pd.read_csv("grade2.csv")
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic_increasing
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
0 2 27 Harsh 55 C
1 2 23 Clara 78 B
2 3 33 Tina 82 A
3 3 34 Amy 88 A
4 3 15 Prashant 78 B
5 3 27 Aditya 55 C
6 3 23 Radheshyam 78 B
7 3 11 Bobby 50 D
The 'Marks' column is sorted: FalseAlso, if a column is sorted in descending order, the is_monotonic_increasing attribute evaluates to False.
import pandas as pd
df=pd.read_csv("grade2.csv")
df.sort_values(by="Marks",inplace=True,ascending=False)
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic_increasing
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
3 3 34 Amy 88 A
2 3 33 Tina 82 A
1 2 23 Clara 78 B
4 3 15 Prashant 78 B
6 3 23 Radheshyam 78 B
0 2 27 Harsh 55 C
5 3 27 Aditya 55 C
7 3 11 Bobby 50 D
The 'Marks' column is sorted: FalseThe is_monotonic_increasing attribute cannot be used with columns having NaN values. The is_monotonic_increasing attribute always evaluates to False if a column has NaN values. You can observe this in the following example.
import pandas as pd
df=pd.read_csv("grade.csv")
df.sort_values(by="Marks",inplace=True,ascending=True,na_position="last")
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic_increasing
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
6 2 27 Harsh 55.0 C
10 3 27 Aditya 55.0 C
4 2 22 Tom 73.0 B
2 1 14 Sam 75.0 B
5 2 15 Golu 79.0 B
0 1 11 Aditya 85.0 A
8 3 34 Amy 88.0 A
1 1 12 Chris NaN A
3 1 15 Harry NaN NaN
7 2 23 Clara NaN B
9 3 15 Prashant NaN B
11 3 23 Radheshyam NaN NaN
The 'Marks' column is sorted: FalseEven if we put the rows having NaN values at the top of the dataframe, the is_monotonic_increasing attribute will evaluate to False.
import pandas as pd
df=pd.read_csv("grade.csv")
df.sort_values(by="Marks",inplace=True,ascending=True,na_position="first")
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic_increasing
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
1 1 12 Chris NaN A
3 1 15 Harry NaN NaN
7 2 23 Clara NaN B
9 3 15 Prashant NaN B
11 3 23 Radheshyam NaN NaN
6 2 27 Harsh 55.0 C
10 3 27 Aditya 55.0 C
4 2 22 Tom 73.0 B
2 1 14 Sam 75.0 B
5 2 15 Golu 79.0 B
0 1 11 Aditya 85.0 A
8 3 34 Amy 88.0 A
The 'Marks' column is sorted: FalseCheck if a Column Is Sorted in Descending Order in a Pandas Dataframe
To check if a column is sorted in descending order in a pandas dataframe, we will use the is_monotonic_decreasing attribute. The is_monotonic_decreasing attribute evaluates to True if a column is sorted in descending order. You can observe this in the following example.
import pandas as pd
df=pd.read_csv("grade2.csv")
df.sort_values(by="Marks",inplace=True,ascending=False)
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic_decreasing
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
3 3 34 Amy 88 A
2 3 33 Tina 82 A
1 2 23 Clara 78 B
4 3 15 Prashant 78 B
6 3 23 Radheshyam 78 B
0 2 27 Harsh 55 C
5 3 27 Aditya 55 C
7 3 11 Bobby 50 D
The 'Marks' column is sorted: TrueIf a column is unsorted or is sorted in ascending order, the is_monotonic_decreasing attribute evaluates to False as shown below.
import pandas as pd
df=pd.read_csv("grade2.csv")
#df.sort_values(by="Marks",inplace=True,ascending=False)
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic_decreasing
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
0 2 27 Harsh 55 C
1 2 23 Clara 78 B
2 3 33 Tina 82 A
3 3 34 Amy 88 A
4 3 15 Prashant 78 B
5 3 27 Aditya 55 C
6 3 23 Radheshyam 78 B
7 3 11 Bobby 50 D
The 'Marks' column is sorted: FalseThe is_monotonic_decreasing cannot be used with columns having NaN values. The is_monotonic_decreasing attribute always evaluates to False if a column has NaN values. You can observe this in the following example.
import pandas as pd
df=pd.read_csv("grade.csv")
df.sort_values(by="Marks",inplace=True,ascending=False,na_position="last")
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic_decreasing
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
8 3 34 Amy 88.0 A
0 1 11 Aditya 85.0 A
5 2 15 Golu 79.0 B
2 1 14 Sam 75.0 B
4 2 22 Tom 73.0 B
6 2 27 Harsh 55.0 C
10 3 27 Aditya 55.0 C
1 1 12 Chris NaN A
3 1 15 Harry NaN NaN
7 2 23 Clara NaN B
9 3 15 Prashant NaN B
11 3 23 Radheshyam NaN NaN
The 'Marks' column is sorted: FalseEven if we put the rows having NaN values at the top of the dataframe, the is_monotonic_decreasing attribute will evaluate to False.
import pandas as pd
df=pd.read_csv("grade.csv")
df.sort_values(by="Marks",inplace=True,ascending=False,na_position="first")
print("The dataframe is:")
print(df)
temp=df["Marks"].is_monotonic_decreasing
print("The 'Marks' column is sorted:",temp)Output:
The dataframe is:
Class Roll Name Marks Grade
1 1 12 Chris NaN A
3 1 15 Harry NaN NaN
7 2 23 Clara NaN B
9 3 15 Prashant NaN B
11 3 23 Radheshyam NaN NaN
8 3 34 Amy 88.0 A
0 1 11 Aditya 85.0 A
5 2 15 Golu 79.0 B
2 1 14 Sam 75.0 B
4 2 22 Tom 73.0 B
6 2 27 Harsh 55.0 C
10 3 27 Aditya 55.0 C
The 'Marks' column is sorted: FalseSuggested Reading: If you are into machine learning, you can read this article on regression in machine learning. You might also like this article on clustering mixed data types in Python.
Check if a Column Is Sorted in a Dataframe Using the Numpy Module
The numpy module in python provides us with different functions to perform operations on numeric data. One such function is the diff() function. The diff() function takes an iterable object as its input argument and returns an array containing the first-order difference of the array elements as shown in the following example.
import numpy as np
df=pd.read_csv("grade2.csv")
marks=df["Marks"]
print("The Marks column is:")
print(marks)
temp=np.diff(marks)
print("Array returned by diff() is:")
print(temp)Output:
The Marks column is:
0 55
1 78
2 82
3 88
4 78
5 55
6 78
7 50
Name: Marks, dtype: int64
Array returned by diff() is:
[ 23 4 6 -10 -23 23 -28]Here, you can observe that the first-order difference is calculated as the difference between (n+1)th and nth element in the input array. For example, the first element of the output array is the difference between the second element and the first element of the input "Marks" column. The second element in the output array is the difference of the third element and the second element of the "Marks" column.
By observing the output, we can conclude “if the ‘Marks’ column is sorted in ascending order, all the values in the output array will be greater than or equal to 0. Similarly, if the ‘marks’ column is sorted in descending order, all the elements in the output array will be less than or equal to 0.” We will use this conclusion to check if the column is sorted in ascending or descending order.
To check if a column of a pandas dataframe is sorted in ascending order, we will use the following steps.
- First, we will calculate the first-order difference of the specified column. For this, we will pass the column to the
diff()function as an input argument. - After that, we will check if all the elements in the output array are less than or equal to 0. For this, we will use the comparison operator and the
all()method. When we use the comparison operator on a numpy array, we get an array of boolean values. Theall()method, when invoked on an array containing boolean values, returnsTrueif all the elements are True. - If the
all()method returns True, it will conclude that all the elements are sorted in ascending order.
You can observe this in the following example.
import numpy as np
df=pd.read_csv("grade2.csv")
df.sort_values(by="Marks",inplace=True,ascending=True)
marks=df["Marks"]
print("The dataframe is:")
print(df)
temp=np.diff(marks)
print("Array returned by diff() is:")
print(temp)
boolean_array= temp>=0
print("Boolean array is:")
print(boolean_array)
result=boolean_array.all()
if result:
print("The marks column is sorted.")
else:
print("The marks column is not sorted.")Output:
The dataframe is:
Class Roll Name Marks Grade
7 3 11 Bobby 50 D
0 2 27 Harsh 55 C
5 3 27 Aditya 55 C
1 2 23 Clara 78 B
4 3 15 Prashant 78 B
6 3 23 Radheshyam 78 B
2 3 33 Tina 82 A
3 3 34 Amy 88 A
Array returned by diff() is:
[ 5 0 23 0 0 4 6]
Boolean array is:
[ True True True True True True True]
The marks column is sorted.To check if a column is sorted in descending order, we will check if all the elements in the output array of the diff() function are less than or equal to 0. For this, we will use the comparison operator and the all() method. When we use the comparison operator on a numpy array, we get an array of boolean values. The all() method, when invoked on an array containing boolean values, returns True if all the elements are True.
If the all() method returns True, it will conclude that all the elements are sorted in descending order. You can observe this in the following example.
import numpy as np
df=pd.read_csv("grade2.csv")
df.sort_values(by="Marks",inplace=True,ascending=False)
marks=df["Marks"]
print("The dataframe is:")
print(df)
temp=np.diff(marks)
print("Array returned by diff() is:")
print(temp)
boolean_array= temp<=0
print("Boolean array is:")
print(boolean_array)
result=boolean_array.all()
if result:
print("The marks column is sorted.")
else:
print("The marks column is not sorted.")Output:
The dataframe is:
Class Roll Name Marks Grade
3 3 34 Amy 88 A
2 3 33 Tina 82 A
1 2 23 Clara 78 B
4 3 15 Prashant 78 B
6 3 23 Radheshyam 78 B
0 2 27 Harsh 55 C
5 3 27 Aditya 55 C
7 3 11 Bobby 50 D
Array returned by diff() is:
[ -6 -4 0 0 -23 0 -5]
Boolean array is:
[ True True True True True True True]
The marks column is sorted.Check if the Index Column Is Sorted in a Dataframe
To check if the index of a dataframe is sorted in ascending order, we can use the index attribute and the is_monotonic attribute as shown below.
import pandas as pd
df=pd.read_csv("grade2.csv",index_col="Marks")
df.sort_index(inplace=True,ascending=True)
print("The dataframe is:")
print(df)
temp=df.index.is_monotonic
print("The Index is sorted:",temp)Output:
The dataframe is:
Class Roll Name Grade
Marks
50 3 11 Bobby D
55 2 27 Harsh C
55 3 27 Aditya C
78 2 23 Clara B
78 3 15 Prashant B
78 3 23 Radheshyam B
82 3 33 Tina A
88 3 34 Amy A
The Index is sorted: TrueTo check if the index of a dataframe is sorted in ascending order, we can use the index attribute and the is_monotonic_increasing attribute as shown below.
import pandas as pd
df=pd.read_csv("grade2.csv",index_col="Marks")
df.sort_index(inplace=True,ascending=True)
print("The dataframe is:")
print(df)
temp=df.index.is_monotonic_increasing
print("The Index is sorted:",temp)Output:
The dataframe is:
Class Roll Name Grade
Marks
50 3 11 Bobby D
55 2 27 Harsh C
55 3 27 Aditya C
78 2 23 Clara B
78 3 15 Prashant B
78 3 23 Radheshyam B
82 3 33 Tina A
88 3 34 Amy A
The Index is sorted: TrueTo check if the index of a dataframe is sorted in descending order, we can use the index attribute and the is_monotonic_decreasing attribute as shown below.
import pandas as pd
df=pd.read_csv("grade2.csv",index_col="Marks")
df.sort_index(inplace=True,ascending=False)
print("The dataframe is:")
print(df)
temp=df.index.is_monotonic_decreasing
print("The Index is sorted:",temp)Output:
The dataframe is:
Class Roll Name Grade
Marks
88 3 34 Amy A
82 3 33 Tina A
78 2 23 Clara B
78 3 15 Prashant B
78 3 23 Radheshyam B
55 2 27 Harsh C
55 3 27 Aditya C
50 3 11 Bobby D
The Index is sorted: TrueYou need to keep in mind that the is_monotonic attribute, is_monotonic_increasing attribute, and the is_monotonic_decreasing always return False if the index column contains NaN values. Therefore, you cannot use these attributes to check if the index is sorted if the index column contains NaN values.
Conclusion
In this article, we have discussed different ways to check if a column is sorted in a pandas dataframe. For this, we have used the pandas library as well as the numpy module. We have also checked if the index of a pandas dataframe is sorted or not.
To learn more about python programming, you can read this article on dictionary comprehension in python. You might like this article on list comprehension in python too.
I hope you enjoyed reading this article. Stay tuned for more informative articles.
Happy Learning!
The post Check if a Column Is Sorted in a Pandas Dataframe appeared first on PythonForBeginners.com.