In this tutorial, we are going to see how to read Google search results in selenium using python.
Read Google Search Results in Selenium:
This example requires the following Softwares to be installed in your machine.
- Install Python
- Install Selenium
- Install ChromeDriver and Browser
You can follow my previous article about how to install the selenium and kickstart with selenium HelloWorld example to get all prerequisites.
Recommended: How to automate the Facebook Login page
How to Read Google Search Results in Selenium:
Here is the use-case for this example.
- Open Google web browser
- Search for – “top 10 python books”
- Grab all the google search URL results related to the above search
- Print the results on the console
from selenium import webdriver
driver = webdriver.Chrome(executable_path="D:\\softwares\\chromedriver_win\\chromedriver.exe")
driver.get("http://www.google.com")
element = driver.find_element_by_name("q");
element.send_keys("Spring MVC Login form Example");
element.submit();
results = driver.find_elements_by_xpath("//div[@class='g']//div[@class='r']//a[not(@class)]");
for result in results:
print(result.get_attribute("href"))
Code Walkthrough:
- importing
webdriverfrom selenium package webdriver.Chrome()function is used to launch the web browser. for this, we have to provide the chromedriver.exe path as an executable_path=”{chromedriverpath}”diver.get("URL")function loads a web page (http://www.google.com) in the current browser session.driver.find_element_by_name()is a driver function used to finds an element in the web page and returns the element as WebElement. if the element wasn’t found it raises NoSuchElementException exception. In our case, it gives Google Search Bar element.send_keys()is a WebElement function used to send simple key events or form fields. In our case, we are sending “top 10 python books” to the form fields as a search query.submit()is a WebElement function is used to perform the submit event on the element. As a result, google provides matches results on the webpage.driver.find_elements_by_xpath()is a driver function used to find the elements in the webpage using the XPath and it returns the elements as WebElements.- Finally iterating the list of results using for loop.
Output:
As a result, google provided the below 13 results in a first page.
https://realpython.com/best-python-books/
https://hackr.io/blog/best-python-books-for-beginners-and-advanced-programmers
https://www.geeksforgeeks.org/best-books-to-learn-python-for-beginners-and-experts-in-2019/
https://www.guru99.com/best-python-books.html
https://stackabuse.com/the-best-python-books-for-all-skill-levels/
https://www.quora.com/Which-is-the-best-book-for-learning-python-for-absolute-beginners-on-their-own
https://stackabuse.com/the-best-python-books-for-all-skill-levels/
https://www.geeksforgeeks.org/best-books-to-learn-python-for-beginners-and-experts-in-2019/
https://www.quora.com/What-are-the-best-books-on-Python
https://www.guru99.com/best-python-books.html
https://www.electronicshub.org/best-python-books/
https://www.edureka.co/blog/best-books-for-python/FAQs:
1. What is XPath?
In short, we can say, XPath is a major element in the XSLT standard and it is used to navigate through elements and attributes in an XML document.
2. How to find an XML element using XPath?
Open a Chrome browser and go to developer tool using F12 then click on Elements tab and press Ctrl+F (Search). There you could see the below search bar.
Type XPath query in the search bar and then you can see the found elements in the yellow-coloured background.
References:
Happy Learning 🙂
