Googlebot Hits in Google Analytics after Using URL Inspection Tool
The other day I was playing with the new GSC URL inspection tool and found one interesting thing. I decided to shoot a quick post about that. This is not something super new or extraordinary, but still good to understand.
We all know that Google Analytics reports many hits from bots: this system is too easy to manipulate. But normally, GA doesn’t have any info on Googlebot visits unless you specifically configure it to show this data. Here is what Google Help says about that:
But here is what I see in my Google Analytics:
The thing is that when you’re using the URL inspect tool for testing a live URL, Google sends a hit (2 hits in most cases) to your Google Analytics view. You can see it in the real-time reports:
The country is the USA, the source, browser and many other dimensions are (not set).
Here is how these hits are recorded in the server logs:
Note that there are 2 hits from Googlebot, the second one being from Nexus 5X which Google uses to check mobile-friendliness. But in Google Analytics, Device Category still shows “desktop” for both hits.
As you can see from the logs, Google visits the robots.txt file on each live URL inspection. This makes sense as the bot needs to check if indexation is allowed.
What’s about the old Google Search Console?
The old GSC also has a similar tool – Fetch as Google. It also sends 2 hits when you choose ‘Fetch and Render’ (and does not send them if you are only fetching the page).
By default, both hits sent by Googlebot are reported in Google Analytics as “desktop”. But if you choose “Mobile: Smartphone”, you will see that one of the hits is “desktop” and the other one is “mobile”.
What’s in it for me?
Nothing terribly serious. But if you often use the URL inspection tool to test live pages, I would exclude googlebot traffic from your filtered GA view and add a custom segment to filter it out from the past reports. The steps are the same as I described in this post about amazon keyword in Google Analytics, just use “googlebot.com” in the ISP filter.
That’s it! Hope you’ve found this quick note useful.
I cook digital marketing dishes. Take 3 tablespoons of on-page SEO, add 2 pinches of backlinks and sprinkle it all with paid advertising. Season to taste with actionable data from Analytics and bake until golden brown. Serve hot.