1
0 Comments

Use Proxy Server for Web Scraping

https://www.octoparse.com/blog/proxy-server-for-web-scraping/?indie=
Web scraping has become a popular way for collecting web data. However, some website owners are fighting back web scraping through limiting the access rate of any single IP. To reduce the chances of getting blocked, we should try to avoid scraping a website with a single IP address. In this article, we will introduce what is a proxy server and some popular web scrapers that have IP proxy features.

Trending on Indie Hackers
Passed $7k 💵 in a month with my boring directory of job boards 39 comments Reaching $100k MRR Organically in 12 months 32 comments 87.7% of entrepreneurs struggle with at least one mental health issue 14 comments How to Secure #1 on Product Hunt: DO’s and DON'Ts / Experience from PitchBob – AI Pitch Deck Generator & Founders Co-Pilot 11 comments Competing with a substitute? 📌 Here are 4 ad examples you can use [from TOP to BOTTOM of funnel] 10 comments Are you wondering how to gain subscribers to a founder's X account from scratch? 9 comments