1
2 Comments

[Poll] What is your biggest challenge related to web scraping?

by Robert Balazsi

I'd be curious to find out what your biggest challenge is related to web scraping.

What is your biggest challenge related to web scraping?
  1. Coming up with robust element selectors
  2. Scraping AJAX pages
  3. Website's structure updated quite often
  4. Bypassing anti-scraping measures
  5. Getting past geo-blocking
  6. Something else (please comment)
Vote
posted to
Webscraping
 on February 8, 2021
Say something nice to robert_balazsi…
Post Comment
  1. 2

    Selenium does almost everything “well” BUT scraping sites with dynamic content is challenging. Don’t you think?

    xavierlesa
    ·
    3 years ago
    ·
    Reply
    1. 2

      It is indeed, especially for scalability reasons. A headless browser is much slower and more resource-intensive as a HTML parser, so a good practice would be to use headless browsers only if absolutely necessary.

      robert_balazsi
      ·
      3 years ago
      ·
      Reply
Trending on Indie Hackers
I've built a 2300$ a month SaaS out of a simple problem. 19 comments 🔥 Roast My Landing Page 12 comments Where can I buy newsletter ad promos? 8 comments Key takeaways growing MRR from $6.5k to $20k for my design studio 6 comments How would you monetize my project colorsandfonts? 5 comments YouTube? How to start 5 comments