BY CATEGORY▼
VBA CATEGORIES▼
VBA - SCRAPING WEBSITES VIDEOS▼
- Excel VBA Part 46 - Querying Web Pages with Query Tables
- Excel VBA Part 47.1 - Browsing to Websites and Scraping Web Page
- Excel VBA Part 47.2 - Scraping Website Tables and Clicking Links
- Excel VBA Part 47.3 - Internet Explorer vs XML HTTP Request
- Excel VBA Part 47.4 - Logging in to a Website with Windows Security
- Excel VBA Part 47.5 - Basic HTTP GET and POST Requests
- Excel VBA Part 48 - Scraping Multiple Web Pages
- Excel VBA Part 49 - Downloading Files from Websites
- Excel VBA Part 57.1 - Getting Started with Selenium Basic and Google Chrome
- Excel VBA Part 57.2 - Basic Web Scraping with Selenium and Google Chrome
- Excel VBA Part 57.3 - Using Different Web Browsers in Selenium
- Excel VBA Part 57.4 - Finding Web Elements in Selenium
VBA - scraping websites videos | Excel VBA Part 47.5 - Basic HTTP GET and POST Requests
Posted by Andrew Gould on 05 June 2019
This video explains the basic differences between the GET and POST methods and how you can use each method to send data along with an HTTP request. You'll learn about adding a query string to a URL for GET requests and how to send values in the body of a POST request. You'll also see how to ensure the values you pass to a request are properly encoded using the Excel EncodeURL function. Throughout the video you'll also learn the basics of using the Chrome Developer Tools to work out how requests are sent by the Chrome browser which helps when translating this to VBA code.
You can download any files that you need to follow the video here.
You can increase the size of the video:

You can view the video in full screen mode as shown on the left, using the icon at the bottom right of the frame.
You can also increase the quality of the video:

You can improve the resolution of the video using another icon at the bottom right of the frame. This will slow down the connection speed, but increase the display and sound quality. This icon only becomes visible when you start playing the video.
Finally, if nothing happens when you play the video, check that you're not using IE in compatibility view.
Thank you for your very helpful tutorial entitled "Excel VBA Introduction Part 47.5 - Basic HTTP GET and POST Requests." I have run into a problem that stumps me. Following the tutorial, I get the following result:
***************************
<!doctype html>
<html lang="en">
<head>
<base href="/">
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no">
<meta name="mobile-web-app-capable" content="yes">
<title>Profits Run</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:400,500,600,700,800">
<link rel="apple-touch-icon" sizes="60x60" href="/assets/images/favicons/apple-touch-icon.png">
<link rel="icon" type="image/png" sizes="32x32" href="/assets/images/favicons/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="/assets/images/favicons/favicon-16x16.png">
<link rel="manifest" href="/assets/images/favicons/site.webmanifest">
<link rel="mask-icon" href="/assets/images/favicons/safari-pinned-tab.svg" color="#5bbad5">
<meta name="msapplication-TileColor" content="#da532c">
<meta name="theme-color" content="#ffffff">
<link href="app.12d5769f220fe4fa467b.css" rel="stylesheet"></head>
<body>
<div id="root"></div>
<script type="text/javascript" src="vendor-d7a24874.js"></script><script type="text/javascript" src="app-4c8acbc2.js"></script></body>
</html>
***************************
However, when I inspect the elements of the website, I get the information I am looking for. I believe the problem is that the java script is not running, and I have been unable thus far to find out how to make the java script run so that I can scrape the portions of it that interest me.
If you are able to give me a suggestion or a work around, I would be very grateful.
Thank you
Hi, the easiest way to get the javascript to run is to use VBA to open the page in a web browser. We have a few videos on how to do this using Internet Explorer (which you may have already seen). If you’re interested in using Chrome you might like to investigate the Selenium library. We don’t have any resources on Selenium but you can find a basic tutorial here. I hope that helps!