The Wayback Machine - https://web.archive.org/web/20240930175110/https://www.geeksforgeeks.org/pandas-groupby/

Pandas GroupBy

Last Updated : 23 Jul, 2024

Groupby is a pretty simple concept. We can create a grouping of categories and apply a function to the categories. It’s a simple concept but it’s an extremely valuable technique that’s widely used in data science. In real data science projects, you’ll be dealing with large amounts of data and trying things over and over, so for efficiency, we use Groupby concept. Groupby concept is really important because it’s ability to aggregate data efficiently, both in performance and the amount code is magnificent. Groupby mainly refers to a process involving one or more of the following steps they are:

Splitting : It is a process in which we split data into group by applying some conditions on datasets.
Applying : It is a process in which we apply a function to each group independently
Combining : It is a process in which we combine different datasets after applying groupby and results into a data structure

The following image will help in understanding a process involve in Groupby concept.
1. Group the unique values from the Team column

2. Now there’s a bucket for each group

3. Toss the other data into the buckets

4. Apply a function on the weight column of each bucket.

Splitting Data into Groups

Splitting is a process in which we split data into a group by applying some conditions on datasets. In order to split the data, we apply certain conditions on datasets. In order to split the data, we use groupby() function this function is used to split the data into groups based on some criteria. Pandas objects can be split on any of their axes. The abstract definition of grouping is to provide a mapping of labels to group names. Pandas datasets can be split into any of their objects. There are multiple ways to split data like:

obj.groupby(key)
obj.groupby(key, axis=1)
obj.groupby([key1, key2])

Note :In this we refer to the grouping objects as the keys.
Grouping data with one key:
In order to group data with one key, we pass only one key as an argument in groupby function.

Python

# importing pandas module
import pandas as pd 
 
# Define a dictionary containing employee data 
data1 = {'Name':['Jai', 'Anuj', 'Jai', 'Princi', 
                 'Gaurav', 'Anuj', 'Princi', 'Abhi'], 
        'Age':[27, 24, 22, 32, 
               33, 36, 27, 32], 
        'Address':['Nagpur', 'Kanpur', 'Allahabad', 'Kannuaj',
                   'Jaunpur', 'Kanpur', 'Allahabad', 'Aligarh'], 
        'Qualification':['Msc', 'MA', 'MCA', 'Phd',
                         'B.Tech', 'B.com', 'Msc', 'MA']} 
   
 
# Convert the dictionary into DataFrame  
df = pd.DataFrame(data1)
 
print(df)

Now we group a data of Name using groupby() function.

Python

# using groupby function
# with one key

df.groupby('Name')
print(df.groupby('Name').groups)

Output :

Now we print the first entries in all the groups formed.

Python

# applying groupby() function to 
# group the data on Name value. 
gk = df.groupby('Name') 
  
# Let's print the first entries 
# in all the groups formed. 
gk.first()

Output :

Grouping data with multiple keys :
In order to group data with multiple keys, we pass multiple keys in groupby function.

Python

# importing pandas module
import pandas as pd 
 
# Define a dictionary containing employee data 
data1 = {'Name':['Jai', 'Anuj', 'Jai', 'Princi', 
                 'Gaurav', 'Anuj', 'Princi', 'Abhi'], 
        'Age':[27, 24, 22, 32, 
               33, 36, 27, 32], 
        'Address':['Nagpur', 'Kanpur', 'Allahabad', 'Kannuaj',
                   'Jaunpur', 'Kanpur', 'Allahabad', 'Aligarh'], 
        'Qualification':['Msc', 'MA', 'MCA', 'Phd',
                         'B.Tech', 'B.com', 'Msc', 'MA']} 
   
 
# Convert the dictionary into DataFrame  
df = pd.DataFrame(data1)
 
print(df)

Now we group a data of “Name” and “Qualification” together using multiple keys in groupby function.

Python

# Using multiple keys in
# groupby() function
df.groupby(['Name', 'Qualification'])

print(df.groupby(['Name', 'Qualification']).groups)

Output :

Grouping data by sorting keys :
Group keys are sorted by default using the groupby operation. User can pass sort=False for potential speedups.

Python

# importing pandas module
import pandas as pd

# Define a dictionary containing employee data
data1 = {'Name': ['Jai', 'Anuj', 'Jai', 'Princi',
                  'Gaurav', 'Anuj', 'Princi', 'Abhi'],
         'Age': [27, 24, 22, 32,
                 33, 36, 27, 32], }


# Convert the dictionary into DataFrame
df = pd.DataFrame(data1)

print(df)

Now we apply groupby() using sort

Python

# using groupby function
# using sort

df.groupby(['Name']).sum()

Output :

Now we apply groupby() without using sort in order to attain potential speedups

Python

# using groupby function
# without using sort

df.groupby(['Name'], sort = False).sum()

Output :

Grouping data with object attributes :
Groups attribute is like dictionary whose keys are the computed unique groups and corresponding values being the axis labels belonging to each group.

Python

# importing pandas module
import pandas as pd 
 
# Define a dictionary containing employee data 
data1 = {'Name':['Jai', 'Anuj', 'Jai', 'Princi', 
                 'Gaurav', 'Anuj', 'Princi', 'Abhi'], 
        'Age':[27, 24, 22, 32, 
               33, 36, 27, 32], 
        'Address':['Nagpur', 'Kanpur', 'Allahabad', 'Kannuaj',
                   'Jaunpur', 'Kanpur', 'Allahabad', 'Aligarh'], 
        'Qualification':['Msc', 'MA', 'MCA', 'Phd',
                         'B.Tech', 'B.com', 'Msc', 'MA']} 
   
 
# Convert the dictionary into DataFrame  
df = pd.DataFrame(data1)
 
print(df)

Now we group data like we do in a dictionary using keys.

Python

# using keys for grouping
# data

df.groupby('Name').groups

Output :

Iterating through groups

In order to iterate an element of groups, we can iterate through the object similar to itertools.obj.

Python

# importing pandas module
import pandas as pd 
 
# Define a dictionary containing employee data 
data1 = {'Name':['Jai', 'Anuj', 'Jai', 'Princi', 
                 'Gaurav', 'Anuj', 'Princi', 'Abhi'], 
        'Age':[27, 24, 22, 32, 
               33, 36, 27, 32], 
        'Address':['Nagpur', 'Kanpur', 'Allahabad', 'Kannuaj',
                   'Jaunpur', 'Kanpur', 'Allahabad', 'Aligarh'], 
        'Qualification':['Msc', 'MA', 'MCA', 'Phd',
                         'B.Tech', 'B.com', 'Msc', 'MA']} 
   
 
# Convert the dictionary into DataFrame  
df = pd.DataFrame(data1)
 
print(df)

Now we iterate an element of group in a similar way we do in itertools.obj.

Python

# iterating an element
# of group

grp = df.groupby('Name')
for name, group in grp:
    print(name)
    print(group)
    print()

Output :

Now we iterate an element of group containing multiple keys

Python

# iterating an element
# of group containing 
# multiple keys

grp = df.groupby(['Name', 'Qualification'])
for name, group in grp:
    print(name)
    print(group)
    print()

Output :
As shown in output that group name will be tuple

Selecting a groups

In order to select a group, we can select group using GroupBy.get_group(). We can select a group by applying a function GroupBy.get_group this function select a single group.

Python

# importing pandas module
import pandas as pd 
 
# Define a dictionary containing employee data 
data1 = {'Name':['Jai', 'Anuj', 'Jai', 'Princi', 
                 'Gaurav', 'Anuj', 'Princi', 'Abhi'], 
        'Age':[27, 24, 22, 32, 
               33, 36, 27, 32], 
        'Address':['Nagpur', 'Kanpur', 'Allahabad', 'Kannuaj',
                   'Jaunpur', 'Kanpur', 'Allahabad', 'Aligarh'], 
        'Qualification':['Msc', 'MA', 'MCA', 'Phd',
                         'B.Tech', 'B.com', 'Msc', 'MA']} 
   
 
# Convert the dictionary into DataFrame  
df = pd.DataFrame(data1)
 
print(df)

Now we select a single group using Groupby.get_group.

Python

# selecting a single group

grp = df.groupby('Name')
grp.get_group('Jai')

Output :

Now we select an object grouped on multiple columns

Python

# selecting object grouped
# on multiple columns

grp = df.groupby(['Name', 'Qualification'])
grp.get_group(('Jai', 'Msc'))

Output :

Applying function to group

After splitting a data into a group, we apply a function to each group in order to do that we perform some operation they are:

Aggregation : It is a process in which we compute a summary statistic (or statistics) about each group. For Example, Compute group sums ormeans
Transformation : It is a process in which we perform some group-specific computations and return a like-indexed. For Example, Filling NAs within groups with a value derived from each group
Filtration : It is a process in which we discard some groups, according to a group-wise computation that evaluates True or False. For Example, Filtering out data based on the group sum or mean

Aggregation :
Aggregation is a process in which we compute a summary statistic about each group. Aggregated function returns a single aggregated value for each group. After splitting a data into groups using groupby function, several aggregation operations can be performed on the grouped data.
Code #1: Using aggregation via the aggregate method

Python

# importing pandas module
import pandas as pd 

# importing numpy as np
import numpy as np
 
# Define a dictionary containing employee data 
data1 = {'Name':['Jai', 'Anuj', 'Jai', 'Princi', 
                 'Gaurav', 'Anuj', 'Princi', 'Abhi'], 
        'Age':[27, 24, 22, 32, 
               33, 36, 27, 32], 
        'Address':['Nagpur', 'Kanpur', 'Allahabad', 'Kannuaj',
                   'Jaunpur', 'Kanpur', 'Allahabad', 'Aligarh'], 
        'Qualification':['Msc', 'MA', 'MCA', 'Phd',
                         'B.Tech', 'B.com', 'Msc', 'MA']} 
   
 
# Convert the dictionary into DataFrame  
df = pd.DataFrame(data1)
 
print(df)

Now we perform aggregation using aggregate method

Python

# performing aggregation using
# aggregate method

grp1 = df.groupby('Name')

grp1.aggregate(np.sum)

Output :

Now we perform aggregation on agroup containing multiple keys

Python

# performing aggregation on
# group containing multiple
# keys
grp1 = df.groupby(['Name', 'Qualification'])

grp1.aggregate(np.sum)

Output :

Applying multiple functions at once :
We can apply a multiple functions at once by passing a list or dictionary of functions to do aggregation with, outputting a DataFrame.

Python

# importing pandas module
import pandas as pd 

# importing numpy as np
import numpy as np
 
# Define a dictionary containing employee data 
data1 = {'Name':['Jai', 'Anuj', 'Jai', 'Princi', 
                 'Gaurav', 'Anuj', 'Princi', 'Abhi'], 
        'Age':[27, 24, 22, 32, 
               33, 36, 27, 32], 
        'Address':['Nagpur', 'Kanpur', 'Allahabad', 'Kannuaj',
                   'Jaunpur', 'Kanpur', 'Allahabad', 'Aligarh'], 
        'Qualification':['Msc', 'MA', 'MCA', 'Phd',
                         'B.Tech', 'B.com', 'Msc', 'MA']} 
   
 
# Convert the dictionary into DataFrame  
df = pd.DataFrame(data1)
 
print(df)

Now we apply a multiple functions by passing a list of functions.

Python

# applying a function by passing
# a list of functions

grp = df.groupby('Name')

grp['Age'].agg([np.sum, np.mean, np.std])

Output :

Applying different functions to DataFrame columns :
In order to apply a different aggregation to the columns of a DataFrame, we can pass a dictionary to aggregate .

Python

# importing pandas module
import pandas as pd 

# importing numpy as np
import numpy as np
 
# Define a dictionary containing employee data 
data1 = {'Name':['Jai', 'Anuj', 'Jai', 'Princi', 
                 'Gaurav', 'Anuj', 'Princi', 'Abhi'], 
        'Age':[27, 24, 22, 32, 
               33, 36, 27, 32], 
        'Address':['Nagpur', 'Kanpur', 'Allahabad', 'Kannuaj',
                   'Jaunpur', 'Kanpur', 'Allahabad', 'Aligarh'], 
        'Qualification':['Msc', 'MA', 'MCA', 'Phd',
                         'B.Tech', 'B.com', 'Msc', 'MA'],
        'Score': [23, 34, 35, 45, 47, 50, 52, 53]} 
   
 
# Convert the dictionary into DataFrame  
df = pd.DataFrame(data1)
 
print(df)

Now we apply a different aggregation to the columns of a dataframe.

Python

# using different aggregation
# function by passing dictionary
# to aggregate
grp = df.groupby('Name')

grp.agg({'Age' : 'sum', 'Score' : 'std'})

Output :

Transformation :
Transformation is a process in which we perform some group-specific computations and return a like-indexed. Transform method returns an object that is indexed the same (same size) as the one being grouped. The transform function must:

Return a result that is either the same size as the group chunk
Operate column-by-column on the group chunk
Not perform in-place operations on the group chunk.

Python

# importing pandas module
import pandas as pd 

# importing numpy as np
import numpy as np
 
# Define a dictionary containing employee data 
data1 = {'Name':['Jai', 'Anuj', 'Jai', 'Princi', 
                 'Gaurav', 'Anuj', 'Princi', 'Abhi'], 
        'Age':[27, 24, 22, 32, 
               33, 36, 27, 32], 
        'Address':['Nagpur', 'Kanpur', 'Allahabad', 'Kannuaj',
                   'Jaunpur', 'Kanpur', 'Allahabad', 'Aligarh'], 
        'Qualification':['Msc', 'MA', 'MCA', 'Phd',
                         'B.Tech', 'B.com', 'Msc', 'MA'],
        'Score': [23, 34, 35, 45, 47, 50, 52, 53]} 
   
 
# Convert the dictionary into DataFrame  
df = pd.DataFrame(data1)
 
print(df)

Now we perform some group-specific computations and return a like-indexed.

Python

# using transform function
grp = df.groupby('Name')
sc = lambda x: (x - x.mean()) / x.std()*10
grp.transform(sc)

Output :

Filtration :
Filtration is a process in which we discard some groups, according to a group-wise computation that evaluates True or False. In order to filter a group, we use filter method and apply some condition by which we filter group.

Python

# importing pandas module
import pandas as pd 

# importing numpy as np
import numpy as np
 
# Define a dictionary containing employee data 
data1 = {'Name':['Jai', 'Anuj', 'Jai', 'Princi', 
                 'Gaurav', 'Anuj', 'Princi', 'Abhi'], 
        'Age':[27, 24, 22, 32, 
               33, 36, 27, 32], 
        'Address':['Nagpur', 'Kanpur', 'Allahabad', 'Kannuaj',
                   'Jaunpur', 'Kanpur', 'Allahabad', 'Aligarh'], 
        'Qualification':['Msc', 'MA', 'MCA', 'Phd',
                         'B.Tech', 'B.com', 'Msc', 'MA'],
        'Score': [23, 34, 35, 45, 47, 50, 52, 53]} 
   
 
# Convert the dictionary into DataFrame  
df = pd.DataFrame(data1)
 
print(df)

Now we filter data that to return the Name which have lived two or more times .

Python

# filtering data using
# filter data
grp = df.groupby('Name')
grp.filter(lambda x: len(x) &gt;= 2)

Output :

A

ABHISHEK TIWARI 13

News

Improve

Previous Article

How to add one row in an existing Pandas DataFrame?

Grouping Rows in pandas

Similar Reads

Creating a Pandas DataFrame

In the real world, a Pandas DataFrame will be created by loading the datasets from existing storage, storage can be SQL Database, CSV file, and Excel file. Pandas DataFrame can be created from the lists, dictionary, and from a list of dictionary etc. A Dataframe is a two-dimensional data structure, i.e., data is aligned in a tabular fashion in rows

Boolean Indexing in Pandas

In boolean indexing, we will select subsets of data based on the actual values of the data in the DataFrame and not on their row/column labels or integer locations. In boolean indexing, we use a boolean vector to filter the data. Boolean indexing is a type of indexing that uses actual values of the data in the DataFrame. In boolean indexing, we can

Python Pandas - DataFrame.copy() function

Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric python packages. Pandas is one of those packages and makes importing and analyzing data much easier. There are many ways to copy DataFrame in pandas. The first way is a simple way of assigning a dataframe object to a variable, but this h

Dealing with Rows and Columns in Pandas DataFrame

A Data frame is a two-dimensional data structure, i.e., data is aligned in a tabular fashion in rows and columns. We can perform basic operations on rows/columns like selecting, deleting, adding, and renaming. In this article, we are using nba.csv file. Dealing with Columns In order to deal with columns, we perform basic operations on columns like

Apply a function to each row or column in Dataframe using pandas.apply()

Applying a function to a single or selected columns/rows in one go is a better way. For this we use the Pandas apply() function. There are different ways to apply a function to each row or column in Pandas DataFrame. We will learn about various ways to Apply Function to Every Row in this article. Creating a Sample DataFrameBefore seeing different w

Python | Pandas tseries.offsets.DateOffset

Dateoffsets are a standard kind of date increment used for a date range in Pandas. It works exactly like relative delta in terms of the keyword args we pass in. DateOffsets work as follows, each offset specifies a set of dates that conform to the DateOffset. For example, Bday defines this set to be the set of dates that are weekdays (M-F). DateOffs

Features of C Programming Language

C is a procedural programming language. It was initially developed by Dennis Ritchie in the year 1972. It was mainly developed as a system programming language to write an operating system. The main features of C language include low-level access to memory, a simple set of keywords, and a clean style, these features make C language suitable for sys

Java IO : Input-output in Java with Examples

Java brings various Streams with its I/O package that helps the user to perform all the input-output operations. These streams support all the types of objects, data-types, characters, files etc to fully execute the I/O operations. Before exploring various input and output streams lets look at 3 standard or default streams that Java has to provide

Java Arithmetic Operators with Examples

Operators constitute the basic building block to any programming language. Java too provides many types of operators which can be used according to the need to perform various calculations and functions, be it logical, arithmetic, relational, etc. They are classified based on the functionality they provide. Here are a few types: Arithmetic Operator

For Loop in Java

Loops in Java come into use when we need to repeatedly execute a block of statements. Java for loop provides a concise way of writing the loop structure. The for statement consumes the initialization, condition, and increment/decrement in one line thereby providing a shorter, easy-to-debug structure of looping. Let us understand Java for loop with

Structures in C++

We often come around situations where we need to store a group of data whether of similar data types or non-similar data types. We have seen Arrays in C++ which are used to store set of data of similar data types at contiguous memory locations.Unlike Arrays, Structures in C++ are user defined data types which are used to store group of items of non

Building Heap from Array

Given an array of N elements. The task is to build a Binary Heap from the given array. The heap can be either Max Heap or Min Heap. Examples: Input: arr[] = {4, 10, 3, 5, 1}Output: Corresponding Max-Heap: 10 / \ 5 3 / \4 1 Input: arr[] = {1, 3, 5, 4, 6, 13, 10, 9, 8, 15, 17}Output: Corresponding Max-Heap: 17 / \ 15 13 / \ / \ 9 6 5 10 / \ / \ 4 8 3

Insertion and Deletion in Heaps

Deletion in Heap:Given a Binary Heap and an element present in the given Heap. The task is to delete an element from this Heap. The standard deletion operation on Heap is to delete the element present at the root node of the Heap. That is if it is a Max Heap, the standard deletion operation will delete the maximum element and if it is a Min heap, i

Association Rule

Association rule mining finds interesting associations and relationships among large sets of data items. This rule shows how frequently a itemset occurs in a transaction. A typical example is a Market Based Analysis. Market Based Analysis is one of the key techniques used by large relations to show associations between items.It allows retailers to

Data encryption standard (DES) | Set 1

This article talks about the Data Encryption Standard (DES), a historic encryption algorithm known for its 56-bit key length. We explore its operation, key transformation, and encryption process, shedding light on its role in data security and its vulnerabilities in today's context. What is DES?Data Encryption Standard (DES) is a block cipher with

Priority Queue in Python

Priority Queues are abstract data structures where each data/value in the queue has a certain priority. For example, In airlines, baggage with the title “Business” or “First-class” arrives earlier than the rest. Priority Queue is an extension of the queue with the following properties. An element with high priority is dequeued before an element wit

Introduction to Internet of Things (IoT) - Set 1

IoT stands for Internet of Things. It refers to the interconnectedness of physical devices, such as appliances and vehicles, that are embedded with software, sensors, and connectivity which enables these objects to connect and exchange data. This technology allows for the collection and sharing of data from a vast network of devices, creating oppor

Language Processors: Assembler, Compiler and Interpreter

Computer programs are generally written in high-level languages (like C++, Python, and Java). A language processor, or language translator, is a computer program that convert source code from one programming language to another language or to machine code (also known as object code). They also find errors during translation. What is Language Proces

Functional Components of a Computer

Computer: A computer is a combination of hardware and software resources which integrate together and provides various functionalities to the user. Hardware are the physical components of a computer like the processor, memory devices, monitor, keyboard etc. while software is the set of programs or instructions that are required by the hardware reso

Basics of Computer and its Operations

Introduction : A computer is an electronic device that can receive, store, process, and output data. It is a machine that can perform a variety of tasks and operations, ranging from simple calculations to complex simulations and artificial intelligence. Computers consist of hardware components such as the central processing unit (CPU), memory, stor

Block Cipher modes of Operation

Encryption algorithms are divided into two categories based on the input type, as a block cipher and stream cipher. Block cipher is an encryption algorithm that takes a fixed size of input say b bits and produces a ciphertext of b bits again. If the input is larger than b bits it can be divided further. For different applications and uses, there ar

Carrier Sense Multiple Access (CSMA)

This method was developed to decrease the chances of collisions when two or more stations start sending their signals over the data link layer. Carrier Sense multiple access requires that each station first check the state of the medium before sending. Prerequisite - Multiple Access Protocols Vulnerable Time: Vulnerable time = Propagation time (Tp)

Generations of Computer

Introduction: A computer is an electronic device that manipulates information or data. It can store, retrieve, and process data. Nowadays, a computer can be used to type documents, send an email, play games, and browse the Web. It can also be used to edit or create spreadsheets, presentations, and even videos. But the evolution of this complex syst

Carry Look-Ahead Adder

The adder produce carry propagation delay while performing other arithmetic operations like multiplication and divisions as it uses several additions or subtraction steps. This is a major problem for the adder and hence improving the speed of addition will improve the speed of all other arithmetic operations. Hence reducing the carry propagation de

goto Statement in C

The C goto statement is a jump statement which is sometimes also referred to as an unconditional jump statement. The goto statement can be used to jump from anywhere to anywhere within a function. Syntax: Syntax1 | Syntax2 ---------------------------- goto label; | label: . | . . | . . | . label: | goto label; In the above syntax, the first line te

Mid-Point Circle Drawing Algorithm

The mid-point circle drawing algorithm is an algorithm used to determine the points needed for rasterizing a circle. We use the mid-point algorithm to calculate all the perimeter points of the circle in the first octant and then print them along with their mirror points in the other octants. This will work because a circle is symmetric about its ce

Unary operators in C

Unary operators are the operators that perform operations on a single operand to produce a new value. Types of unary operatorsTypes of unary operators are mentioned below: Unary minus ( - )Increment ( ++ )Decrement ( -- )NOT ( ! )Addressof operator ( & )sizeof()1. Unary MinusThe minus operator ( - ) changes the sign of its argument. A positive

Difference Between Structure and Union in C

Structures in C is a user-defined data type available in C that allows to combining of data items of different kinds. Structures are used to represent a record. Defining a structure: To define a structure, you must use the struct statement. The struct statement defines a new data type, with more than or equal to one member. The format of the struct

Sliding Window Protocol - Go Back N (GBN)

Sliding Window Protocol is actually a theoretical concept in which we have only talked about what should be the sender window size (1+2a) in order to increase the efficiency of stop-and-wait ARQ. Now we will talk about the practical implementations in which we take care of what should be the size of the receiver window. Practically it is implemente

Data Abstraction and Data Independence

Database systems comprise complex data structures. In order to make the system efficient in terms of retrieval of data, and reduce complexity in terms of usability of users, developers use abstraction i.e. hide irrelevant details from the users. This approach simplifies database design. Level of Abstraction in a DBMSThere are mainly 3 levels of da

Article Tags :

Misc

Practice Tags :

Misc