In this article, you’ll learn about our anti-bot filter, which excludes bot activity from reports.
Some actions in partners’ projects can be performed by bots. Such impressions and clicks are reported and distort one’s view of the conversions of programs and tools as well as possible income and other rates. To make our partner statistics more accurate, we at Travelpayouts made a unique anti-bot filter. It automatically excludes bot traffic from program reports.
Which Traffic is Considered Bot Traffic?
We consider bot traffic to be any automated traffic. Thus, the statistics don’t take into account the bots of search engines (such as Google), parsers, SEO and click-fraud bots, bots for vulnerability search, and others.
We don’t collect all automated traffic, but find the biggest and most popular bots. This helps us eliminate a significant number of bots from reports and show more accurate statistics.
How We Detect Bot Traffic
Every Internet user has a user agent (ID with additional parameters). The agents of large bots are known and stored in public sources.
We created a bot that:
- Collects fresh user agents with suspicious activity from open sources.
- Sends the received data to our database.
- Browses the history of Travelpayouts logs (system information files) and compares user agents in the database and logs.
- If the user agents match, they are marked as bots. These user agents are excluded from reports on partner programs.
In order for partners to get correct statistics, this algorithm is run daily and the data in the reports are updated regularly.