Python - Get list of File

Python - Get List of All Files in Directory and Sub-Directories

To retrieve the list of all files in a directory and its sub-directories, you can use Python's os.walk() function. This function generates an iterator that traverses the directory tree, providing access to directories and files at each level.

In this tutorial, we will explore examples demonstrating how to achieve this efficiently using os.walk(). We will also cover filtering files by specific extensions.

What is os.walk()?

os.walk() generates a tuple for each directory it traverses. Each tuple contains:

  • root: Current directory path.
  • dirs: List of sub-directories in the current directory.
  • files: List of file names in the current directory.

You can iterate through this output to access all directories and files recursively.

Examples

1. List All Files Recursively

This example demonstrates how to list all files in a given directory and its sub-directories.

Python Program

import os

# Specify the directory path
path = "C:/workspace/python"

# Initialize an empty list to store file names
filelist = []

# Traverse the directory tree
for root, dirs, files in os.walk(path):
    for file in files:
        # Add the full path of the file to the list
        filelist.append(os.path.join(root, file))

# Print all file names
for name in filelist:
    print(name)

Explanation

  1. os.walk(path): Iterates through all sub-directories and files starting from the given path.
  2. os.path.join(root, file): Combines the directory path and file name into a full path.
  3. The list filelist accumulates all file paths for later use.

Output

C:\pythonexamples\python-create-directory.png
C:\pythonexamples\python-remove-file.png
C:\pythonexamples\scatter-plot-example.py
C:\pythonexamples\tkinter-example.py
C:\pythonexamples\sample\example.py
C:\pythonexamples\sample\example1.py

2. Filter Files by Extension

In this example, we list only files with the .py extension in the directory and its sub-directories.

Python Program

import os

# Specify the directory path
path = "C:/workspace/python"

# Traverse the directory tree
for root, dirs, files in os.walk(path):
    for file in files:
        # Check for .py extension
        if file.endswith(".py"):
            print(os.path.join(root, file))

Explanation

  • file.endswith(".py"): Filters files that end with the .py extension.
  • os.path.join(): Constructs the full path of each matching file for display or further processing.

Output

C:\pythonexamples\scatter-plot-example.py
C:\pythonexamples\tkinter-example.py
C:\pythonexamples\sample\example.py
C:\pythonexamples\sample\example1.py

3. Count All Files in Directory and Sub-Directories

This example counts the total number of files in a directory and its sub-directories.

Python Program

import os

# Specify the directory path
path = "C:/workspace/python"

# Initialize file counter
file_count = 0

# Traverse the directory tree
for root, dirs, files in os.walk(path):
    file_count += len(files)

print(f'Total number of files: {file_count}')

Explanation

  • len(files): Counts the number of files in the current directory.
  • file_count += len(files): Adds the count of files from each directory to the total.

Output

Total number of files: 6

4. Save List of Files to a Text File

Here, we save the list of all files in a directory and its sub-directories to a text file.

Python Program

import os

# Specify the directory path
path = "C:/workspace/python"

# Open a file to write the file list
with open("file_list.txt", "w") as f:
    for root, dirs, files in os.walk(path):
        for file in files:
            # Write the full file path to the text file
            f.write(os.path.join(root, file) + "\n")

print("File list saved to file_list.txt")

Explanation

  • open("file_list.txt", "w"): Opens a text file in write mode.
  • f.write(): Writes each file's path to the text file.
  • \n: Adds a newline after each file path for readability.

Output

A text file file_list.txt is created containing the paths of all files.

Summary

In this tutorial, we explored various methods to list all files in a directory and its sub-directories using Python. We covered:

  • Listing all files recursively with os.walk().
  • Filtering files by extension.
  • Counting the total number of files.
  • Saving the list of files to a text file.

These techniques are useful for file management tasks in Python and can be extended further based on your needs.

Python Libraries