How to Concatenate DataFrames in Pandas?

Concatenate DataFrames – pandas.concat()

You can concatenate two or more Pandas DataFrames with similar columns. To concatenate Pandas DataFrames, usually with similar columns, use pandas.concat() function.

In this tutorial, we will learn how to concatenate DataFrames with similar and different columns.

Syntax of pandas.concat() method

The syntax of pandas.concat() is:

pandas.concat(objs, axis=0, join='outer', join_axes=None, ignore_index=False, keys=None, levels=None, names=None, verify_integrity=False, sort=None, copy=True)

Examples

1. Concatenate DataFrames with similar columns

In this example, we take two DataFrames with same column names and concatenate them using concat() function.

import pandas as pd
	
df_1 = pd.DataFrame(
	[['Somu', 68, 84, 78, 96],
	['Kiku', 74, 56, 88, 85],
	['Ajit', 77, 73, 82, 87]],
	columns=['name', 'physics', 'chemistry','algebra','calculus'])

df_2 = pd.DataFrame(
	[['Amol', 72, 67, 91, 83],
	['Lini', 78, 69, 87, 92]],
	columns=['name', 'physics', 'chemistry','algebra','calculus'])	

frames = [df_1, df_2]

# Concatenate dataframes
df = pd.concat(frames, sort=False)

# Print dataframe
print("df_1\n------\n",df_1)
print("\ndf_2\n------\n",df_2)
print("\ndf\n--------\n",df)

Output

The two DataFrames are concatenated. But the index is not in order. You can reset the index by using reset_index() function.

import pandas as pd
	
df_1 = pd.DataFrame(
	[['Somu', 68, 84, 78, 96],
	['Kiku', 74, 56, 88, 85],
	['Ajit', 77, 73, 82, 87]],
	columns=['name', 'physics', 'chemistry','algebra','calculus'])

df_2 = pd.DataFrame(
	[['Amol', 72, 67, 91, 83],
	['Lini', 78, 69, 87, 92]],
	columns=['name', 'physics', 'chemistry','algebra','calculus'])	

frames = [df_1, df_2]

# Concatenate dataframes
df = pd.concat(frames)

# Reset index
df.reset_index(drop=True, inplace=True)

# Print dataframe
print(df)

Output

   name  physics  chemistry  algebra  calculus
0  Somu       68         84       78        96
1  Kiku       74         56       88        85
2  Ajit       77         73       82        87
3  Amol       72         67       91        83
4  Lini       78         69       87        92

2. Concatenate two DataFrames with different columns

In this following example, we take two DataFrames. The second dataframe has a new column, and does not contain one of the column that first dataframe has.

pandas.concat() function concatenates the two DataFrames and returns a new dataframe with the new columns as well. The dataframe row that has no value for the column will be filled with NaN short for Not a Number.

import pandas as pd
	
df_1 = pd.DataFrame(
	[['Somu', 68, 84, 78, 96],
	['Kiku', 74, 56, 88, 85],
	['Ajit', 77, 73, 82, 87]],
	columns=['name', 'physics', 'chemistry','algebra','calculus'])

df_2 = pd.DataFrame(
	[['Amol', 72, 67, 91, 83],
	['Lini', 78, 69, 87, 92]],
	columns=['name', 'physics', 'chemistry','geometry','calculus'])	

frames = [df_1, df_2]

# Concatenate dataframes
df = pd.concat(frames, sort=False)

# Print dataframe
print("df_1\n------\n",df_1)
print("\ndf_2\n------\n",df_2)
print("\ndf\n--------\n",df)

Output

Summary

In this tutorial of Python Examples, we learned how to concatenate one or more DataFrames into a single DataFrame, with the help of well detailed examples.