Python Soup - Find Elements by CSS Selector

Python BeautifulSoup - Find Elements by CSS Selector

To select elements in a HTML page by CSS selector using BeautifulSoup, you can use the select() method, which allows you to use a CSS selector.

In this tutorial, we shall go through a step by step process to find elements in a webpage using a CSS selector string.

Steps to find elements by CSS selector

  1. Import BeautifulSoup from bs4 library.
  2. Given HTML content as a string in html_content.
  3. Parse HTML content using BeautifulSoup() constructor, and store the returned object in soup.
  4. Call select() method on the soup object, and pass the CSS selector string as argument. The method returns a list of Tag objects. These elements in the list are the HTML elements that match the given CSS selector.

Example: Get Elements with the CSS selector 'div.article'

In the following program, we take a sample HTML content, and then find the elements with given CSS selector 'div.article' using BeautifulSoup.select() method.

Python Program

from bs4 import BeautifulSoup

# Sample HTML content
html_content = """
<html>
    <body>
        <div class="article">Article 1</div>
        <div class="article">Article 2</div>
        <div class="story">Story 1</div>
        <div class="story">Story 2</div>
    </body>
</html>
"""

# Parse the HTML content
soup = BeautifulSoup(html_content, "html.parser")

# Find elements using CSS selector string
elements = soup.select("div.article")

for element in elements:
    print(element)

Output

<div class="article">Article 1</div>
<div class="article">Article 2</div>

The css selector is 'div.article', which means all the div elements with the class article. There are two elements with the given CSS selector, and they are printed to the output.

Summary

In this Python BeautifulSoup tutorial, we have seen how to find the elements in a HTML page by CSS selector using BeautifulSoup select() method, with a step by step process, and an example.