Why Ad-Verification Solutions Should Not Sample Impressions

Introduction - How Sampling is Currently Used in Ad Verification

Sampling is used to estimate a characteristic of a larger population. Sampling is an appropriate method when direct measurement of the entire population is overly burdensome or impossible to do within the required time. Sampling is not appropriate in cases where every item in a data set can be measured quickly and cheaply.

Ad-verification vendors sample impressions for two reasons:

  1. The length of the verification process is longer than the auction cycle of selling an ad-impression. In this case, analyzing 100% of impressions would significantly hinder or prevent ad-delivery by causing timeouts* and non-renders.

  2. Verification methods rely on computationally expensive behavioral checks. To reduce costs, vendors measure a subset of supply to provide acceptable pricing to clients.

*In all cases of time-outs, advertisers lose opportunities to reach consumers. If an advertiser is not on ad-server billing, they will be paying for each non-render event.

Types of Users

Internet users can be divided into three categories, human users (48%), “good” bots (23%), and “bad” bots (29%). Publishers create value and generate revenue by selling their online real estate to advertisers, who pay for access to human users’ eyes. Advertisers lose when those ads are displayed to bots, both good and bad.

“Good” bots have numerous legitimate uses. For example, good bots include search engine crawlers that are critical for keeping the internet running smoothly. Good bots primarily operate from data centers, as opposed to human-operable machines.

If websites blocked search engine bots, their content would disappear from search results and the website would lose many human users overnight. If search engine crawlers stopped visiting websites, the search engine would lose its value. If your website shows up in a Google search, it is only because the Googlebot has visited your site. Therefore, it is necessary for these bots to visit websites and the digital media industry must adapt to find a way to prevent ads from being served to them.

“Bad” bots include content scrapers, headless browsers, botnets, and other unwanted visitors to a site. Like good bots, they consume advertising and server resources, but they offer no benefit in return. Bad bot traffic can originate from data centers or human-operable devices. Effective ad-tech partners prevent their clients’ media spend from being consumed by “bad” bots.

In summary, advertisers, agencies and publishers must accept that both good and bad bots will be visiting their websites. Those serving the interests of advertisers must focus on how to measure and prevent the delivery of ads to both good and bad bots. Each ad viewed by a bot is a waste of advertiser funds and publisher resources. Advertisers must not be billed for impressions viewed by good or bad bots.

The Problem: How Advertisers Lose When Using Sampled Data

Imagine the following scenario:

  1. An advertiser’s agency enlists a verification vendor to monitor its digital media spend. The vendor monitors a $1M campaign and samples 10% of impressions. Of those 10%, the advertiser measured 25% bot traffic (fraudulent).

  2. The advertiser extrapolates the data and determines that 25% of their $1M campaign was spent on waste, and asks for a refund of $250k from its DSP.

  3. The DSP refuses to refund the advertiser until it speaks with the SSP’s and Exchanges in its supply chain. The SSP’s states that the verification vendor’s sample cannot be guaranteed to be representative of the entire campaign.

  4. The DSP also tells the advertiser that impressions are not all the same cost. 25% of impressions being fraudulent does not mean 25% of spend was wasted. The 25% waste could be on low CPM impressions, and the 75% on high CPM impressions. Therefore, a 25% refund could be excessive.

  5. The advertiser does not have the data to refute these two points, and accepts the waste as “the cost of doing business”.

The Solution

Method believes the following must be true to protect the interests of advertisers and avoid the above scenario:

  1. Verification vendors only provide actionable analytics when verifying every impression.

  2. Advertisers must have rapid access to full receipts of campaign spend (data for every impression).

As shown by the example above, only complete monitoring can be used to recover funds spent on waste. Analytics on every impression (cost, domain, IP, human/bot) are required to calculate the exact amount wasted advertiser spend. But before advertisers can be refunded for previous waste and prevent waste in future campaigns, they must first obtain this data.

Adtech Won’t Fix Ad Fraud Because It Is Too Lucrative

Every hour of every day a million little crimes are committed online. And every time it happens, hundreds of legitimate businesses all over the world, with boards and shareholders and mission statements — some of them publicly listed — put the proceeds of those crimes in their own pockets. They do so knowingly. 

Some people might consider it extraordinary that technology businesses — businesses that claim they can discern the intent of one buyer from a billion in milliseconds — somehow can’t recognize when millions of ads a month are served to a single unique user ID in, for instance, Belarus.

Method Media Intelligence's Shailin Dhar states that "when you study actual bot traffic in isolation as well as in live ad exchanges, you realize quickly that the truth can be described best as ‘take the industry estimates of the financial impact of fraud and multiply by ten.’”

Method Media Intelligence to Present at 6th Prague Marketing Festival in March 2019

The 6th Prague Marketing Festival will be from March 20th to 22nd, 2019. Ticket sales began on June 5th, 2018. Method Media Intelligence's Shailin Dhar and Praneet Sharma will be presenting, following up on their seminar on ad fraud from earlier this year.

Pixalate Says it Uncovered ‘Sophisticated’ Android Ad Fraud Potentially Worth $75 Million Annually

Ad safety company Pixalate has uncovered what it says is a ‘sophisticated’ mobile app fraud potentially costing advertisers up to $75 million a year if allowed to run unchecked.

The company says it discovered that Android app, MegaCast which allows users to broadcast content to Google’s Chromecast, was using ‘mobile app laundering’ to generate fraudulent ad impressions.

Even Privacy Advocates Are Tracking You Online

Each time someone visits CAPrivacy.org, software gleans what information it can about them, then sends that information to Facebook, including their IP address, what web pages they were on before and after visiting, and so on. At this point, both the visitor and the website have basically lost control of what happens with that information. 

Brands And Agencies Are ‘Resistant’ To Solving Viewability And Ad Fraud, Say Researchers

Ad fraud and invalid traffic are the topics of this piece from Andrew Birmingham of Which 50.

MMI's Shailin Dhar stated that "for all the promises that digital data promotes, there still remains a gap in understanding the systems that generate them. The open, complex systems of the internet seem to be shrouded behind abstractions of what is marketed and what is actually possible."

Praneet Sharma added that “as the funders of this system, it’s ironic that advertisers have so little share of this data ‘currency’. For this system to make good on all its offerings and capabilities, the brands must be given full transparency of each and every penny spent, and gain a full awareness of its limitations". 

Method Media Intelligence has created Proactive Auditing to address current challenges in digital advertising. Proactive Auditing provides full receipts of digital campaigns by tracking every impression. A dashboard provides clients with real time access to campaign metrics to identify wasted spend and enable clawbacks and rapid optimization.

 

 

Kerrisdale Capital Hit On QuinStreet A Harbinger For Ad Arbitragers In An Age Of Growing Transparency - Which50

Read commentary from Which 50's Andrew Birmingham about the Kerrisdale Capital report on QuinStreet. Kerrisdale Capital is a New York based hedge fund which focuses on short activism. 

Andrew Birmingham interviewed Shailin Dhar of Method Media Intelligence for his view on QuinStreet. “Re-brokering and arbitrage are the main pillars of affiliate and performance marketing. The main reasons are that if you can create an image palatable enough for actual advertisers, you can go out and find 100 individual parties that will provide small amounts of traffic that can be aggregated to look like an attractive level to advertisers.", Shailin said. 

In addition, he said, “advertisers will slowly but surely get wind of the fact that the party they purchase leads from usually has zero idea how they were generated. Lead scrubbing is essential; it was my first encounter with ad fraud in 2011.”

Ad Fraud Interview With Focus Agency

Read Shailin Dhar's interview with Petr Michl of Focus Agency. This interview follows Method Media Intelligence's presentation at the 2017 Prague Marketing Festival. 

Shailin commented on how ad fraud has become such an easy way to generate illegitimate revenue, and marketers are currently behind the curve on preventing wasted spend. Last year US advertisers were estimated to spend $7-8B delivering ads to robotic visitors, while less than $0.2B was spent on ad fraud prevention. 

Method Media Intelligence serves advertisers by detecting invalid impressions and working with agencies to account for the use of advertising dollars. Proactive auditing is one tool MMI uses to empower advertisers to generate a higher return on advertising spend. 

Ad Fraud off the Charts

In the April 1st AdContrarian newsletter, Praneet Sharma of Method Media Intelligence points out the main reason that fraud estimates are widely inaccurate.

In December, CNBC reported $237B would be spent on digital advertising in 2018, and the Wall Street Journal reported 28% of internet traffic showed "non-human signals". Using this projection, $66.4B of advertising spend will be spent on fraud this calendar year. 

Since fraudsters are one step ahead, it is reasonable to assume there is some undetected fraud. Praneet believes that "avoiding detection will be the major obstacle that fraudsters will present."

The Advertising Industry Needs to Balance out Its Programmatic Transparency Quest

The Drum’s digital editor Ronan Shields reflects on some of the key talking points of The Drum’s Programmatic Punch, NYC, conference hosted earlier this week. Method Media Intelligence's Shailin Dhar was on the ad fraud panel, discussing how advertisers and agencies can win the fight against fraud in this industry.

Shailin added that “fraud is not 5%, it’s more like 50%, when you look at the amount of non-human traffic on the web, you’d be looking at almost 60% [of total traffic], the fact that we’re pretending [it's not that big of a problem] is just bizarre".

Fake McDonald’s Video Campaign Reveals How Widely The “Benefits” Of Ad Fraud Are Distributed

This article from Andrew Birmingham of Which 50 discusses the results of a fake McDonald's video ad campaign. Shailin Dhar of Method Media Intelligence explains how fraud originates and argues that ad tech vendors need to adjust their detection methods by exonerating human users.

Bots Who Stole Christmas & Ad Fraud, MetaX, Method Media Intelligence, NYC

This post contains a link to Method Media Intelligence's presentation on ad fraud at Blockchain Xplore event for MetaX at Microsoft Center in New York City.

How to Tell When Someone Is Talking BS About Blockchain - The Drum

Blockchain has become one of the most overused buzzwords in many industries. In this article from Rebecca Stewart of The Drum, the claims and solutions related to the applications of blockchain technology are examined.

Method Media Intelligence's Shailin Dhar believes that "a lack of clarity and misinformation in the marketplace are the two biggest challenges holding blockchain back", and "everyone would benefit from creating education cycles in their organizations to make sure their teams have a sufficient understanding of what the word blockchain means to them and the environment they operate in".

Method Media Intelligence to Present at Programmatic Punch on March 28th in New York City

Shailin Dhar of Method Media Intelligence will be presenting at Programmatic Punch on March 28th in New York City. Programmatic Punch is a one-day event by The Drum focused on changes in the digital advertising industry. 

Shailin Dhar and Mike Zaneis, CEO of TAG, will be the ad fraud panel members. Specifically, the speakers will discuss how ad fraud effects every aspect of marketing and the damage that is done to brands when they appear against inappropriate content. 

Lessons on Preventing Ad Fraud and How to Seal the Cracks - The Drum

Read an interview between Shailin Dhar and Danielle Gibson of The Drum. In this interview, Shailin describes how current industry practices allow ad fraud to continue.

In addition, Shailin details MMI's "exoneration method", which starts with the assumption that traffic is invalid and seeks to validate, using a "guilty until proven innocent" approach to detection. In addition, MMI's traffic quality metrics are determined by measuring 100% of impression events as opposed to sampling only a fraction of a whole data set. 

For more, read the full interview.

Method Media Intelligence to Present at Blockchain Xplore on February 27th in New York City

Praneet Sharma and Shailin Dhar from Method Media Intelligence will be presenting at MetaX's Blockchain Xplore event at the Microsoft Technology Center in New York City on February 27th, 2018. 

Blockchain Xplore is a one-day, educational conference designed to drive a deeper understanding of the potentials of blockchain-based technology for the digital advertising industry.

The title of their talk is "The Bots Who Stole Christmas".

Google AdWords Allows Targeting Using Phone Numbers, Addresses - AdAge

Read this article from AdAge, which details how "Google has expanded the consumer-targeting capabilities of its AdWords platform to let marketers use phone numbers and mailing addresses to reach their right people."

According to Method Media Intelligence's Praneet Sharma, this is a standard capability in the advertising industry. 

Inside Adform’s ‘Hyphbot’ ad fraud takedown - TheDrum

Read this article from Ronan Shields of The Drum, in which the operation of "Hyphbot" is described. Hyphbot is the name of a botnet discovered by Adform, estimated to be generating revenues of $500,000/day. Adform's CRO, Jay Stevens, called Hyphbot a "call to arms" for the adtech industry.

Adform consulted MMI for peer review prior to the publication of their findings. 

Method Media Intelligence to Present an Intro to Digital Ads and Ad Fraud at the RocketSpace Campus

Method Media Intelligence will present an Intro to Digital Ads and Ad Fraud at the RocketSpace campus to corporate members. At this event, Shailin Dhar and Praneet Sharma will give an overview of the monetization of the internet, and an in-depth look into how user attention and user data get fed into the ecosystem of online advertising. 

RSVP for the event here.

Shailin Dhar and Praneet Sharma Presented at the 5th Prague Marketing Festival

Shailin Dhar and Praneet Sharma presented at the 5th Prague Marketing Festival on Friday, November 10th. The event brought together 1,700 marketers and 9 speakers from around the world to discuss current challenges in digital marketing. 

Shailin and Praneet covered the basics of ad fraud: what it is, why it happens, how it is done, and what individuals can do to help stop it.

Read their presentation here.