How to get all the links in webpage in Selenium Python?

Get all the Links in Webpage in Selenium Python

To get all the links present in the webpage using Selenium in Python, call find_elements() method on the webdriver object, and pass By.TAG_NAME as selector, and “a” for value.

driver.find_elements(By.TAG_NAME, 'a')

The above method call returns all the link elements in the webpage as a list, List[WebElement] object.

Example

In this example, we shall consider loading the HTML file at path https://pythonexamples.org/tmp/selenium/index-32.html . The contents of this HTML file is given below.

<html>
 <body>
  <p>Hello World! Read <a id="about_link" href="/tmp/selenium/index-31-about.html">About Hello World</a>.</p>
  <br>
  <p>Latest Articles</p>
  <ul>
    <li><a id="new_article" href="/tmp/selenium/index-31-new-article.html">About New Article</a></li>
  </ul>
 </body>
</html>
Copy
How to get all the links in webpage in Selenium Python?

Now, we shall get all the links (anchor elements) with the tag name as a. And print them to standard output.

Python Program (Selenium)

from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By

# Setup chrome driver
driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))

# Navigate to the url
driver.get('https://pythonexamples.org/tmp/selenium/index-32.html')

# Find all link elements
links = driver.find_elements(By.TAG_NAME, 'a')

# Iterate over link elements
for link in links:
    print(link.get_attribute('outerHTML'))

# Close the browser
driver.quit()

Output

<a id="about_link" href="/tmp/selenium/index-31-about.html">Hello World</a>
<a id="new_article" href="/tmp/selenium/index-31-new-article.html">New Article 1</a>

Summary

In this Python Selenium tutorial, we learned how to get all the HTML link elements in a webpage, using find_elements() method in Selenium.

Related Tutorials

Quiz on Selenium

Q1. What is the method used to find web elements in Selenium Python?

Not answered

Q2. What is the method used to interact with web elements in Selenium Python?

Not answered

Q3. What is the method used to navigate back to the previous page in Selenium Python?

Not answered

Q4. What is the method used to select a value from a drop-down list in Selenium Python?

Not answered

Q5. What is the method used to clear the text from an input field in Selenium Python?

Not answered
Code copied to clipboard successfully 👍