Crawler architecture: Avoid getting requests counted in Google Analytics

Question

We run a service where we need to index our clients's website once in a while. One of our clients has just noticed, that the traffic from our bot is being seen in their analytics (they use Google Tag Manager). Of cause they could try and filter it, however I'm wondering why it gets tracked in the first place, since we do not have JS enabled in the bot, and they have already ticked “Exclude traffic from known bots and spiders”. Two questions:

1. Can we, from our side (the bot), do anything to tell google analytics NOT to count in the traffic?
2. If not, how can they then filter out our bot? We have set a user agent, but it doesn't seem to be available in google analytics - neither is our IP.

Answer 1

I am thinking about why is this tracked, if there is Google Tag Manager and you have disabled JS.

No JS

And the result seems to be noscript variant of tagmaneger included via iframe.

<noscript><iframe src="//www.googletagmanager.com/ns.html?id=GTM-XXXX"
height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>

So try to block URL: www.googletagmanager.com in your crawler.

With JS

Just set window variable:

window['ga-disable-UA-XXXX-Y'] = true;