22 Aug
2013

Update: Bing Ads Crawler Interfering with Google Analytics: Possible Click-fraud?

For any of you advertising with Bing Ads and using Google Analytics, I need you to do me a favor. Run this advanced segment on any and all clients you have who are advertising with Bing Ads in conjunction with All Traffic:

https://www.google.com/analytics/web/template?uid=K5lxvkj2T6aPnVMShKCBJQ

Particularly, run it for the past 60 days and make sure to note any discrepancies if there are any. When I ran it for a client from July 15th through August 21st, I have a difference of more than 13,500 visits.

MSN Crawler Discrepancy

 

If you see something similar, this is an issue.

It was brought to my attention by Brian Massey, one of the conversion rate optimizers at ConversionScientist.com. They’re great at what they do, so they were a little disconcerted when they noticed their bounce rate skyrocket. They were able to narrow it down in Google Analytics to one particular browser called “Mozilla Compatible Agent.” Looking at the browser versions and the network domain, visitors from this “Mozilla Compatible Agent” was showing a 100% bounce rate to the tune of some 180,000 visits.

99% Bounce Rate! WTF?

 

This looks to be what is clearly a crawler issue coming from msn.com as a network domain. For the uninitiated, the Network Domain reveals the domain of a users ISP. One should expect to see Comcast.com, Cox.com, rr.com, etc. Major and minor ISPs alike. The fact that this one ISP is sending tens of thousands of visits with a 100% bounce rate fits the pattern of crawler traffic, which Google Analytics hasn’t automatically filtered out.

The fix here that Brian and I came up with was to add a filter that removes traffic from the Network Domain msn.com.

Apply this Filterv! Or Don't. Whatevs.

 

Seems like an easy enough solution. I went to double check to make sure any of my clients weren’t having the same issue. I ran into one client and noticed they were seeing the same traffic from this crawler. As curious as I am, I wanted to see which pages were being hit the most with this new crawler.

When I added “Landing Pages” as the secondary dimension, I saw a list of URLs with marketing scripts on them.

All the Marketing Scripts

 

These marketing scripts for this particular client are only used on PPC ads. I hadn’t noticed this on any of my other clients as this is the only client I have running Bing Ads. Just for verification, I reached back out to Brian if his client was using Bing Ads, and he confirmed that they were. I spent the rest of the day contacting other companies I knew that were running Bing Ads to see if this was happening. I created the MSN ISP advanced segment and shared it with colleagues at other agencies to have them run it on any client using Bing Ads. This crawler has been sending widespread traffic for the past 60 days.

As this was going on, my client reached out to their PPC agency asking them if they were seeing this. This is the response that they sent that I am printing with permission. I have only removed the name of the agency:

As far as we can tell the issue is because of the bug they’re seeing in their systems, which seems to have been resolved (at least temporarily) by adding funds into the account.  We’ll certainly be checking that closely to see if it goes down again.  Other clients with prepay accounts have had the same issue, and received the same feedback from their Yahoo reps: that their is a bug and to fix it add funds.

According to the agency, this is a bug they have and are seeing, and to fix it is to add funds to the account. What doesn’t sit well with me is why would adding funds fix crawler traffic? This tells me one of three things:

  1. Depleted accounts are prone to having their ads crawled by the MSN.com bot.
  2. This is legitimate traffic that isn’t being redirected and processed properly either by Bing Ads, a content network, or by Google Analytics.
  3. These clicks from a crawler of some sort are actually counting in Bing Ads, costing companies advertising dollars.

Let’s take a look at each scenario:

1. Depleted accounts are prone to having their ads crawled by the MSN.com bot.

In this case, really there’s very little to do but apply the filter referenced above. Just scrub the traffic from the GA profile going forward. It’s a new crawler that the guys at Google haven’t added yet. It’s a pain, but the issue is solved.

2. This is legitimate traffic that isn’t being redirected and processed properly either by Bing Ads, a content network, or by Google Analytics.

Keep in mind, I think the first scenario is correct, but this is a very likely candidate to be true. As practitioners, most of us know that Display Ads and Content Network ads get lots of impressions and a few clicks. And sometimes, they can come with a high bounce rate. Years ago it freaked me the eff out when my click-through-rates plummeted the first time I added someone to a PPC content network (IT’S GONNA KILL MY QUALITY SCORES!).

I don’t think this is far fetched, but let’s say for whatever reason a display network or a content network doesn’t process properly. Similar to a shortened link, or an app that strips out UTM parameters from the URL, this could be a case of an ad network not processing legitimate traffic. In this case, adding a filter to remove traffic from the network domain MSN.com would actually be scrubbing legitimate traffic to the website so the problem isn’t actually solved.

This leads me to the third and most controversial possibility.

3. These clicks from a crawler of some sort are actually counting in Bing Ads, costing companies advertising dollars. 

I hate saying this because it makes me sound alarmist, and I only want to be if there is legitimate concern. But the agency response that my client got doesn’t sit well with me. Let’s revisit:

…the issue is because of the bug they’re seeing in their systems, which seems to have been resolved (at least temporarily) by adding funds into the account.  

How in the world is a bug fixed by adding funds to an account?

I strongly lean in the direction that the first scenario is the correct answer. But replenishing spent accounts to fix a bug they have on their end doesn’t jibe. The rep is saying “to fix it, spend more money.” This just gives me a bad feeling that  some of this traffic is actually counting as clicks in Bing. A crawler would be coming through and triggering numerous ads and costing clients money.

I really hate to say it, but that sounds like click-fraud. Massive click-fraud on a scale so wide I can’t with good conscience say that it is actual click-fraud. But if the solution is to send them more money, then the client is throwing money to fix a problem that no one knows the actual answer to. This bug or whatever, even if it’s not directly causing clicks on ads, is still causing advertisers to spend more money just…because.

The only thing I can recommend is just at least see if any clients of yours that are running Bing Ads campaigns, is to at least use the advanced segment to check to see if they have the same issue. I’ll leave it up to you to add a filter to remove the traffic going forward. I’m still hesitant because I want to know for sure what the issue is, or until someone has a better answer than “spend more money to fix it.”

I’ll provide more updates if I get any.

 

Update

I got an update from Brian Massey last week, and wanted to make sure it was up:

“I got some Bing data from my client. Here’s a scatterplot of their Bing click data vs. the Bing visits reported by GA data with your Exclude MSN ISP segement enabled. There’s a good correlation here (86%), so I don’t think we’re losing Bing clicks by filtering out the MSN.com ISP visits. This is for July 23 – Aug 22.”

MSN Bot Crawling vs Clicks

 

“I say filter that traffic!”

I’d say he’s probably right and it’s okay to go and do that. I’ve been seeing similar correlations on my end, which would strongly lead to it being okay to filter. I just don’t like the agency’s explanation of the whole thing.

  • disqus_bKvgcH8Q2B

    thank you so much for posting this. tons of advertisers i know have this problem – this is the only solution i found that worked.

    • http://ManofActionMetrics.com/ Nate Griffin

      Very happy to help out. I imagine it’s something that will be fixed over time, but for now this will at least help you out before then.

  • Andrew Vignuzzi

    We had the same issue (including the payment problem which required manual intervention by the reps). I noticed that all of our extra Bing traffic was to advertising landing pages with a single visit and an immediate bounce. Our rep told us to filter on Mozilla Compatible Agent. We did that, and subsequently lost all of the Bing traffic in our filtered profiles. I think your suggestion of filtering on msn.com is the right choice as I am seeing similar correlation between Bing paid clicks and GA reported Search Alliance visits when I only remove msn.com. Thanks for posting this. I wish Google and Bing would just get along.

    • http://ManofActionMetrics.com/ Nate Griffin

      I had initially tried the “Mozilla Compatible Agent” filter too, but when I dug down to the network domain, I felt filtering that would be more accurate. “Mozilla Compatible Agent” isn’t just used by crawlers. And remember, this is “Network Domain” or “ISP Domain” we are filtering, not referring domain. That has seemed to throw some people off too.

      As far as playing nice together, I think for the most part they are trying to play as nicely as possible considering Bing and Google are competitors. I’m still more interested in seeing if other pay-for solutions are seeing the same issue, because at least that would mean it’s a general issue and not just two competitors having a measuring contest.

  • Jessica Carr

    Question – I’m seeing a ton of extra visits from the domain msn.com but the browser is Internet Explorer. But, it does show only 1 page visit and visit duration of 0. My visits are more than double the clicks I’m seeing in Bing ads. Do you think this msn.com domain is the same issue you’re talking about even though the browser is Internet Explorer?

    Thanks!

    • http://ManofActionMetrics.com/ Nate Griffin

      It sounds exactly like the same issue, just with a different browser type. That’s why I used the filter at the network domain. Since MSN isn’t a consumer facing ISP (like Comcast or Charter), that was the only thing I could figure to filter that wouldn’t remove legitimate traffic.

      If it is that, then use that filter up top and it will start removing that traffic.

      • Jessica Carr

        Thanks, Nate! This was super helpful. My client was freaking out that their Bing conversion rate was going down and she was basing it on visits and I just couldn’t figure it out because our click CVR was improving!

        • http://ManofActionMetrics.com/ Nate Griffin

          Are you using the UTM parameters in Bing ads? Check what GA says under the campaigns tab.

          If you need an official explanation from Bing Ads to take to your client, check out my follow up post here: http://manofactionmetrics.com/2013/09/06/msn-crawler-its-totes-not-click-fraud/

          • Jessica Carr

            Yes, we are using utm params.

            Thanks for that follow up article as well!