I want to scrape round about 10-15 millions of keywords from google every month.
My problem is the IP-Ban that occurs every 100-200 Keywords.
I've tested some methods to avoid the IP-Ban and want to discuss with you, what is the best Method for scraping google.
Methods I've testes are:
– VPN-Crawler with HideMyAs, that reconnect on ervery ban.
– DSL – Crawling with a crawler that reconnects on every ban.
So what are your prefered methods to crawl tons of keywords?
I've heread about…