Start Here
Stories
Ideas
Start Here
Learn what it means to be an indie hacker.
Community
Connect with other founders and creators.
Stories
Get inspired by others' success.
Ideas
Get your creative juices flowing.
Products
See what everyone's working on.
Store
Buy an Indie Hackers t-shirt.
Newsletter
Stay up-to-date in 5 minutes or less.
Log In
Sign Up
1
Like
0
Bookmarks
2
Comments
Report
[Poll] What is your biggest challenge related to web scraping?
by
Robert Balazsi
I'd be curious to find out what your biggest challenge is related to web scraping.
What is your biggest challenge related to web scraping?
Coming up with robust element selectors
Scraping AJAX pages
Website's structure updated quite often
Bypassing anti-scraping measures
Getting past geo-blocking
Something else (please comment)
Vote
Trending on Indie Hackers
I've built a 2300$ a month SaaS out of a simple problem.
19 comments
🔥 Roast My Landing Page
12 comments
Where can I buy newsletter ad promos?
8 comments
Key takeaways growing MRR from $6.5k to $20k for my design studio
6 comments
How would you monetize my project colorsandfonts?
5 comments
YouTube? How to start
5 comments
Selenium does almost everything “well” BUT scraping sites with dynamic content is challenging. Don’t you think?
It is indeed, especially for scalability reasons. A headless browser is much slower and more resource-intensive as a HTML parser, so a good practice would be to use headless browsers only if absolutely necessary.