Python Beautiful Soup Module
Beautiful Soup is a Python library for pulling data out of HTML and XML files. It works with your favorite parser to provide idiomatic ways of navigating, searching, and modifying the parse tree.
Installation
pip install beautifulsoup4Importing
from bs4 import BeautifulSoupParsing HTML
To parse a document, pass it into the BeautifulSoup constructor.
html_doc = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
"""
soup = BeautifulSoup(html_doc, 'html.parser')Navigating the Tree
print(soup.title)
# <title>The Dormouse's story</title>
print(soup.title.string)
# The Dormouse's story
print(soup.p)
# <p class="title"><b>The Dormouse's story</b></p>Searching the Tree
find() and find_all()
find() returns the first match, find_all() returns a list of matches.
# Find all <a> tags
links = soup.find_all('a')
for link in links:
print(link.get('href'))
# Find by ID
link2 = soup.find(id="link2")CSS Selectors
You can use select() to find elements using CSS selectors.
# Select by class
soup.select(".sister")
# Select by ID
soup.select("#link1")
# Nested selection
soup.select("p.story a")