Last week I re-activated my old advertisement system on one of our company’s high traffic sites. And after one day I understood that the number of views and clicks are not reasonable, I mean they were ten times higher than I expected. When I checked the raw site logs, unexpectedly, I found about 80% of the site traffic belonged to search engine robots and crawlers!!! Not only famous American ones like Google, Bing, and Yahoo but also Russian Yandex, Chinese Baidu, and Iranian Yooz were very active on the site. There were two solutions for solving the problem: 1. Blocking search engine traffic and preventing their robots and crawlers to indexing my site, an expert knows this action means suicide
