Bash Remove Duplicates in Array

Bash Remove Duplicates in Array

In Bash scripting, removing duplicates from an array is useful for various tasks that require ensuring unique elements in an array.

Syntax

unique_array=()
for element in "${array[@]}"; do
    if [[ ! " ${unique_array[@]} " =~ " $element " ]]; then
        unique_array+=("$element")
    fi
done

The basic syntax involves using a for loop to iterate over the elements of the array and conditionally add unique elements to a new array.

Example Bash Remove Duplicates in Array

Let's look at some examples of how to remove duplicates from an array in Bash:

1. Remove Duplicates from an Array of Strings

This script initializes an array with duplicate string elements, removes duplicates, and prints the unique array.

#!/bin/bash

array=("apple" "banana" "apple" "cherry" "banana")
unique_array=()
for element in "${array[@]}"; do
    if [[ ! " ${unique_array[@]} " =~ " $element " ]]; then
        unique_array+=("$element")
    fi
done
echo "Unique array: ${unique_array[@]}"

In this script, the array variable array is initialized with the elements 'apple', 'banana', 'apple', 'cherry', and 'banana'. The for loop iterates over each element in the array, and the if statement checks if the element is not already in the unique_array. If true, the element is added to the unique_array. The script then prints the unique array.

Remove duplicates from an array of strings in Bash

2. Remove Duplicates from an Array of Numbers

This script initializes an array with duplicate numeric elements, removes duplicates, and prints the unique array.

#!/bin/bash

array=(1 2 2 3 1 4)
unique_array=()
for element in "${array[@]}"; do
    if [[ ! " ${unique_array[@]} " =~ " $element " ]]; then
        unique_array+=("$element")
    fi
done
echo "Unique array: ${unique_array[@]}"

In this script, the array variable array is initialized with the elements 1, 2, 2, 3, 1, and 4. The for loop iterates over each element in the array, and the if statement checks if the element is not already in the unique_array. If true, the element is added to the unique_array. The script then prints the unique array.

Remove duplicates from an array of numbers in Bash

3. Remove Duplicates from an Array with Elements from User Input

This script initializes an array with elements from user input, removes duplicates, and prints the unique array.

#!/bin/bash

array=()
while true; do
    read -p "Enter an element (or 'done' to finish): " element
    if [ "$element" == "done" ]; then
        break
    fi
    array+=("$element")
done

unique_array=()
for element in "${array[@]}"; do
    if [[ ! " ${unique_array[@]} " =~ " $element " ]]; then
        unique_array+=("$element")
    fi
done
echo "Unique array: ${unique_array[@]}"

In this script, the array variable array is initialized as an empty array using (). The user is prompted to enter elements, which are added to the array using the += operator. When the user types 'done', the loop exits. The for loop iterates over each element in the array, and the if statement checks if the element is not already in the unique_array. If true, the element is added to the unique_array. The script then prints the unique array.

Remove duplicates from an array with elements from user input in Bash

Conclusion

Removing duplicates from an array in Bash is a fundamental task for ensuring unique elements in an array in shell scripting. Understanding how to remove duplicates can help you manage and manipulate arrays effectively in your scripts.