Python videos | Python Part 15 - Scraping websites

Posted by Andy Brown on 04 November 2021


This tutorial covers all aspects of extracting data from websites using Python: from the ethics and legality of web scraping, via the structure of a web page's document object model through to using the Python requests and BeautifulSoup modules to extract meaning from HTML.

This video is currently only available to YouTube channel members in the appropriate tier. If you’re already a channel member, you can watch the video on YouTube; otherwise, click here to see the perks of becoming a channel member.

If you're not a channel member, don't worry - the video will be released publicly after a short period of member exclusivity.

This video has the following accompanying files:

File name Type Description
wyndham.htm HTML page The example web page for this tutorial
wiseowl-logo.png Image Image referenced by example web page
Useful websites.txt Text file Useful website addresses
The requests module.py Python code Using the requests module
HTML from file.py Python code Getting HTML from a file
Basic scraping.py Python code Basic web scraping using BeautifulSoup
Possible parsers.png Image Possible parsers for BeautifulSoup
Scraping 1- chaining elements copy.py Python code Chaining HTML tags
Scraping 2- navigable strings copy.py Python code Getting visible text using navigable strings
Scraping 3 - relatives.py Python code Children descendants and other relatives
Scraping 3 - relatives.png Image The possible relatives you can reference
Scraping 4 - finding.py Python code Using findall to get matching elements
Scraping 5 - CSS selectors.py Python code Using select with CSS selectors

Click to download a zipped copy of the above files.

After watching this video, you may like to test your understanding by doing one or more of the following exercises:

You can also download the answers to each exercise from the links above.

Making a  video bigger

You can increase the size of your video to make it fill the screen like this:

View full screen

Play your video (the icons shown won't appear until you do), then click on the full screen icon which appears as shown at its bottom right-hand corner.

 

When you've finished viewing a video in full screen mode, just press the Esc key to return to normal view.

Improving the quality of a video

To improve the quality of a video, first click on the Settings icon:

Settings icon

Make sure you're playing your video so that the icons shown appear, then click on this gear icon at the bottom right-hand corner.

 

Choose to change the video quality:

Video quality

Click on Quality as shown to bring up the submenu.

 

The higher the number you choose, the better will be your video quality (but the slower the connection speed):

Connection speed

Don't choose the HD option unless you have a fast enough connection speed to support it!

 

Is your Wise Owl speaking too slowly (or too quickly)?  You can also use the Settings menu above to change your playback speed.

This page has 0 threads Add post