Google Trends Scraper
Scrape Google Trends to gather all the information you need!
PROBLEM
STATEMENT
The client had scaling issues with their existing scraper and needed something more robust to extract the required data from Google Trends. We were required to implement an algorithm to counter the blocking issues and retrieve a large volume of information.
SOLUTION
We devised a multiple-layer strategy to improve the scaling of the scraper and resolve the blocking issue. The scraper was integrated with multiple API providers (including our customized API written in Playwright), to provide a strong backup for retrieving the information. It allowed us to extract the data even if one or two of our providers failed to respond in the desired manner.
Input
List of keywords to be tracked and the time period for their tracking
Output
JSON showing trends of given keywords over time
Tools &
Technologies
Node.js (Express.js)
Apify
Yagmail (for emailing reports)
