How can I scroll down a webpage using Selenium with VBA?
Sub Going_down() Dim driver As New WebDriver Dim posts As Object, post As Object driver.Start "chrome", "http://fortune.com/fortune500" driver.get "/list/" driver.wait 500 for x = 0 to 200 driver.ExecuteScript("window.scrollTo(0, document.body.scrollHeight);") driver.wait 500 next x Set posts = driver.FindElementsByXPath("//li[contains(concat(' ', @class, ' '), ' small-12 ')]") For Each post In posts i = i + 1 Cells(i, 1) = post.FindElementByXPath(".//a").Attribute("href") Next post End Sub
I would just call their API instead of using selenium. BTW Why are you scraping all these data?
Thanks for your response. The data of that site more specifically any site is not important to me at all. However, what does important to me is that whatever the structure of a site is, I can deal with that. That is why I rove different sites and try to understand their pattern. That's it.