- VIDEOS HOME PAGE
- .NET (14)
- Business Intelligence (40)
- Integration Services (19)
- Macros and Programming (82)
- Microsoft Excel (70)
- Microsoft Office (92)
- Miscellaneous (1)
- Power BI (35)
- Power Platform (35)
- Python (31)
- Report Builder (107)
- Reporting Services (113)
- SQL (42)
- Visual Basic for Applications (215)
- Visual C# (14)
- Excel VBA - Basics (24)
- VBA User Forms (22)
- Excel VBA - pivot tables (9)
- Excel VBA - charts (6)
- VBA - advanced (14)
- VBA - working with files (12)
- VBA - linking applications (12)
- VBA - working with Outlook (14)
- Built-in VBA functions (9)
- VBA - working with data (57)
- VBA - scraping websites (25)
- VBA - working with shapes (5)
- VBA - classes and structures (6)
VBA - SCRAPING WEBSITES VIDEOS▼
- Excel VBA Part 46 - Querying Web Pages with Query Tables
- Excel VBA Part 47.1 - Browsing to Websites and Scraping Web Page
- Excel VBA Part 47.2 - Scraping Website Tables and Clicking Links
- Excel VBA Part 47.3 - Internet Explorer vs XML HTTP Request
- Excel VBA Part 47.4 - Logging in to a Website with Windows Security
- Excel VBA Part 47.5 - Basic HTTP GET and POST Requests
- Excel VBA Part 48 - Scraping Multiple Web Pages
- Excel VBA Part 49 - Downloading Files from Websites
- Excel VBA Part 57.1 - Getting Started with Selenium Basic and Google Chrome
- Excel VBA Part 57.2 - Basic Web Scraping with Selenium and Google Chrome
- Excel VBA Part 57.3 - Using Different Web Browsers in Selenium
- Excel VBA Part 57.4 - Finding Web Elements in Selenium
- Excel VBA Part 57.5 - Implicit and Explicit Waits in Selenium
- Excel VBA Part 57.6 - Working with Multiple Tabs in Selenium
- Excel VBA Part 57.7 - Using Select Drop Down Lists in Selenium
- Excel VBA Part 57.8 - Printing in Google Chrome using Selenium
- How do I open Google Chrome in full screen mode using Selenium in VBA?
- Is it possible to start Google Chrome hidden using Selenium in VBA?
- How do I access elements in the Shadow DOM using Selenium in VBA?
- How do I Refer to a Parent Element in Selenium for VBA?
- How do I hold Shift and click a link in Selenium for VBA?
- How do I scroll down a web page using Selenium in VBA?
- How do I copy CSV data from a website using VBA?
- How do I open Google Chrome with a user profile using VBA?
- How do I use Google Translate with Excel VBA?
VBA - scraping websites videos | How do I access elements in the Shadow DOM using Selenium in VBA?
Posted by Andrew Gould on 03 April 2021
There are no files which go with this video.
There are no exercises for this video.
Making a video bigger
You can increase the size of your video to make it fill the screen like this:
Play your video (the icons shown won't appear until you do), then click on the full screen icon which appears as shown at its bottom right-hand corner.
When you've finished viewing a video in full screen mode, just press the Esc key to return to normal view.
Improving the quality of a video
To improve the quality of a video, first click on the Settings icon:
Make sure you're playing your video so that the icons shown appear, then click on this gear icon at the bottom right-hand corner.
Choose to change the video quality:
Click on Quality as shown to bring up the submenu.
The higher the number you choose, the better will be your video quality (but the slower the connection speed):
Don't choose the HD option unless you have a fast enough connection speed to support it!
Is your Wise Owl speaking too slowly (or too quickly)? You can also use the Settings menu above to change your playback speed.
Chrome: Version 96.0.4664.93 (Offizieller Build) (32-Bit)
The goal is to get out the actual path of the downloads directory for chrome
I can’t access the “#shadow-root” part.
If I understand it correct, I need loop to each of the 5 different “#shadow-root” levels
I believe that Chrome v96 has introduced a change which affects the ExecuteScript method in SeleniumBasic. I'm having similar issues with manipulating the print preview window.
I'm afraid that I don't know of a solution at the moment, sorry.
Do you have any workaround for the Chrome 96 + issue while trying to locate elements inside Shadow root.
I am unable to find the solution to this problem anywhere online.
Thanks in advance..!!
Hi! Sorry no, I haven't found a solution to this yet. This thread indicates that the official Selenium release has solved this from v4.1. The VBA implementation of Selenium isn't officially supported so it may be that this will never be resolved. Annoying.
I've watched all of the 57 series of videos several times and have learned a great deal and continue to come back to them when i get stuck, however finding this shadow host is stumping me and I haven't found an answer here or anywhere else on the net. So I thought I might go to the source and ask the guru himself.
Here's what i've tried stepping through(F8) the macro to ensure everything is loaded with Selenium Basic of a google chrome pdf with a shadow dom. I'm after a table that is in the chrome browser window that after my macro runs I have to Ctrl Copy and then run another macro to organize and format the table data as I need it. It's a secure website and the rest of the macro runs fine opening 5 tabs in the browser finding all of the elements I need but I'd like to automate this last repetetive task so I would not have to stop Ctrl C and V the data into excel and run another macro. The HTML is:
<pdf-viewer id="viewer"></pdf-viewer> flex
<div id="container"> flex
<embed id="plugin" type="application/x-google-chrome-pdf" ...> '<-After this
Dim ShadowHost As Selenium.WebElement
Dim ShadowRoot As Selenium.WebElement
Set ShadowHost = .FindElement(FindBy.XPath("/html/body/pdf-viewer"))
Set ShadowHost = .FindElement(FindBy.Css("#viewer"))
Both the above tried seperatly give:
Run-Time error 7
Element not found for XPath=/html/body/pdf-viewer
'I tried this and didn't get an error:
Set ShadowHost = .FindElementByXPath("/html/body/*")
'BUT the Locals window says ShadowHost tagname = "embed"
'which is the table I'm after!
'but using devtools find gives:
<pdf-viewer id="viewer"></pdf-viewer> flex == $0
Then I run the following:
Set ShadowRoot = .ExecuteScript(Script:="retutn arguments.ShadowRoot", arguments:=ShadowHost"))
'and i get:
Run-time error '17':
'So I tried:
Set ShadowRoot = .ExecuteScript(Script:="retutn arguments.ShadowRoot", arguments:=.FindElementByXPath("/html/body/*")"))
'and got the same error 17
'The element has no other shadow-root above but I can't select anything but body.
'I've tried js ch.querySelector("#viewer")
'and a bunch more but can't get the ShadowHost selected.
'So i thought ok I'll just Ctrl A, Ctrl C, .getclipboard like this:
Set ShadowHost = .FindElementByTag("body").SendKeys(.keys.Control, "a")
Set ShadowHost = .FindElementByTag("body").SendKeys(.keys.Control, "c")
Set dataObj = New MSForms.DataObject
'Gives me Compile error mismatch
'So tried this:
Run-Time error "59":
Unrecognized tag name.
'which is what the local window says the tagname of the WebElement is.
So in conclusion, i have 4 questions:
Why the errors:
Compile error mismatch?
ok 5 questions:
Any ideas as to what's going on and can you help a faithful but frustrated follower?
Thanks for any of the questions you might answer,
PS: There is very little on the net on Ctrl copy and paste, using Action or Actions class (do these 2 classes exist in Selenium Basic?)
a question on one's own very frustrating. Would love to see a video with the Selenium Basic keyboard commands. Guess this counts as 6 questions.
That's a lot of questions!
I believe that working with a PDF is significantly different to working with regular web pages. The pdf-viewer element doesn't appear to be exposed to the DOM (as you've discovered!).
It seems that there are two main ways to do this (neither of which I've attempted):
2) Download the PDF file and use the Acrobat VBA library to manipulate it
Sorry I can't be any more help than that but I hope it points you in the right direction!