- Aug 30, 2012
- 6,598
Great article by Paul Ducklin
Adblockers do pretty much what they say.
They usually run as browser plugins, so they can see what’s coming and going in your browser, try to identify ads, and stop them from being downloaded, rendered or displayed.
You can also block ads at your web gateway, if you have one, but the idea is the same: let through the bulk of the site, but get rid of the ads.
Adblockers can recognise ads in numerous ways, for example:
The principle is generic: write an algorithm which examines data objects and divides them into two distinct sets, X and not-X.
ADBLOCKERS CONSIDERED HARMFUL
In theory, adblockers ought to be uncontroversial.
Some countries block the sites you are allowed to view (by law, in practice, or both), but we don’t know of any jurisdictions where you aren’t allowed to filter your own traffic by choice, over and above any minimum required by law.
But in practice, adblockers have turned into a contentious issue, because many sites that allow free access rely on ad revenue as their way of recovering what we’ll refer to as “the cost of free.”
As a result, people who use adblockers are seen as leeches, for want of a better word, who enjoy free content while suppressing any chance of the website making money out of ads.
Indeed, anti-adblocking site PageFair, in a joint press release with Adobe in August 2015, claimed that adblocking would cost the business world an astonishing $22,000,000,000 (yes, that’s 22 billion dollars!) in 2015.
But there’s a deeper aspect to this dilemma.
ADS CONSIDERED HARMFUL
Although lots of users block ads simply because they don’t like them (which makes you wonder just how much ad revenue they would generate if they were compelled to see the ads, but that’s a question for another time), we know that many people block ads for security reasons.
That’s because of malvertising, where crooks hack into an ad server’s delivery network, insert malware, and sit back while mainstream sites start attacking their own visitors with poisoned ads.
If your site serves ads chosen from 100 different ad providers, and each ad provider has 100 different ads in its current active database, even one poisoned ad will end up distributed widely, but only occasionally, making it hard to track down and deal with.
Worse still, malvertising often appears in websites that you are inclined to trust: high-profile victims in the past year have included the Daily Mail and Forbes.
Ironically, PageFair, having said in its August press release that “it is tragic that [adblock] users are inadvertently inflicting multi-billion dollar losses on the very websites they most enjoy”, was itself the victim of malvertising at the end of October 2015.
One response from ad networks is to detect that you’re using an adblocker, and then block you in return, treating the ads as a sort of subscription: if you unblock ads on the site, you’re deemed to have paid your admission fee, and you’ll be allowed back in.
Think of this as adblocker blocking.
But now, a Californian adblocker blocker is going one step further, and offering an adblocker bypass.
THE ADBLOCKER BYPASS
According to online marketing site Marketing Land, the process goes something like this.
There’s a bypass loader and a bypass proxy.
The loader tries to fetch an ad conventionally, and checks to see if it turns up in the browser.
If not, the loader figures that it has spotted an adblocker, because something is getting in the way of loading the ad.
So the loader scrambles its own ad-fetching JavaScript code, obfuscates the URL from where the ad will be fetched, and tries again.
This time, instead of connecting to the ad server directly, the new and scrambled ad-fetcher goes via a bypass proxy, resulting in deliberately-disguised JavaScript issuing a deliberately-disguised ad request via a deliberately-disguised site.
Of course, if an adblocker can spot known ad servers using easily-updated technology such as a blocklist, it can detect known proxies via its blocklist too.
The ad proxies are, in effect, just ad servers with a different name.
So the bypass proxy gets obfuscated too, for example by using a domain generation algorithm to switch server names every so often, and by changing, or “fluxing”, DNS entries so that the browser proxies move around on the internet.
The bypass proxy then fetches the desired ad from the ad server, and rewrites its content so that any links to the real ad server that are embedded in the ad are themselves adjusted to go via the proxy.
This rewriting isn’t strictly necessary, because each recursive request to the ad server would go back through the bypass loader, get blocked and thus detected, and then get rewritten to go via the proxy anyway. But rewriting the links inside each ad makes things much faster, because only the first visit to the ad server needs to go through the test-to-see-if-it-will-be-blocked process.
If this sounds like how cybercrooks fight back against security products, with obfuscated JavaScript, disguised URLs, andregularly changing proxy servers and DNS records, don’t be surprised.
The principle is generic: create an algorithm which examines data objects and if they are in set X, rewrites them so that they are in not-X.
WHAT HAPPENS NEXT?
We’re not sure how well this trick is going to work.
We’re not thinking about the technological aspects here, but the cultural ones.
If I’ve blocked your ads – whether I don’t like them, don’t trust them, or both – then blocking me from your site unless I agree to unblock those ads seems perfectly reasonable.
If we can reach a willing buyer/willing seller compromise, then we will both end up happy; if not, then neither of us will end up with something at the expense of the other.
But forcing ads on me, especially if I’ve blocked your ads because of of security concerns and yet you are tricking my browser into displaying them in a way I won’t notice until it’s too late…
…how is that going to win me over?
As a commenter pointed out when we last discussed malvertising:
No, we don’t expect companies to give us everything for free. But neither should companies expect us to sacrifice our safety for their product. It’s a risk evaluation.
Perhaps a better approach would be to set about building an ad network that people were willing to unblock out of choice?
Isn’t that better than giving them an ad network with which they end up playing a cat-and-mouse game of detect-evade-detect-evade?
Adblockers do pretty much what they say.
They usually run as browser plugins, so they can see what’s coming and going in your browser, try to identify ads, and stop them from being downloaded, rendered or displayed.
You can also block ads at your web gateway, if you have one, but the idea is the same: let through the bulk of the site, but get rid of the ads.
Adblockers can recognise ads in numerous ways, for example:
- By maintaining a blocklist of URLs used to link to ads.
- By detecting the JavaScript that is used to fetch ads.
- By spotting the HTML used for the actual ad content.
The principle is generic: write an algorithm which examines data objects and divides them into two distinct sets, X and not-X.
ADBLOCKERS CONSIDERED HARMFUL
In theory, adblockers ought to be uncontroversial.
Some countries block the sites you are allowed to view (by law, in practice, or both), but we don’t know of any jurisdictions where you aren’t allowed to filter your own traffic by choice, over and above any minimum required by law.
But in practice, adblockers have turned into a contentious issue, because many sites that allow free access rely on ad revenue as their way of recovering what we’ll refer to as “the cost of free.”
As a result, people who use adblockers are seen as leeches, for want of a better word, who enjoy free content while suppressing any chance of the website making money out of ads.
Indeed, anti-adblocking site PageFair, in a joint press release with Adobe in August 2015, claimed that adblocking would cost the business world an astonishing $22,000,000,000 (yes, that’s 22 billion dollars!) in 2015.
But there’s a deeper aspect to this dilemma.
ADS CONSIDERED HARMFUL
Although lots of users block ads simply because they don’t like them (which makes you wonder just how much ad revenue they would generate if they were compelled to see the ads, but that’s a question for another time), we know that many people block ads for security reasons.
That’s because of malvertising, where crooks hack into an ad server’s delivery network, insert malware, and sit back while mainstream sites start attacking their own visitors with poisoned ads.
If your site serves ads chosen from 100 different ad providers, and each ad provider has 100 different ads in its current active database, even one poisoned ad will end up distributed widely, but only occasionally, making it hard to track down and deal with.
Worse still, malvertising often appears in websites that you are inclined to trust: high-profile victims in the past year have included the Daily Mail and Forbes.
Ironically, PageFair, having said in its August press release that “it is tragic that [adblock] users are inadvertently inflicting multi-billion dollar losses on the very websites they most enjoy”, was itself the victim of malvertising at the end of October 2015.
One response from ad networks is to detect that you’re using an adblocker, and then block you in return, treating the ads as a sort of subscription: if you unblock ads on the site, you’re deemed to have paid your admission fee, and you’ll be allowed back in.
Think of this as adblocker blocking.
But now, a Californian adblocker blocker is going one step further, and offering an adblocker bypass.
THE ADBLOCKER BYPASS
According to online marketing site Marketing Land, the process goes something like this.
There’s a bypass loader and a bypass proxy.
The loader tries to fetch an ad conventionally, and checks to see if it turns up in the browser.
If not, the loader figures that it has spotted an adblocker, because something is getting in the way of loading the ad.
So the loader scrambles its own ad-fetching JavaScript code, obfuscates the URL from where the ad will be fetched, and tries again.
This time, instead of connecting to the ad server directly, the new and scrambled ad-fetcher goes via a bypass proxy, resulting in deliberately-disguised JavaScript issuing a deliberately-disguised ad request via a deliberately-disguised site.
Of course, if an adblocker can spot known ad servers using easily-updated technology such as a blocklist, it can detect known proxies via its blocklist too.
The ad proxies are, in effect, just ad servers with a different name.
So the bypass proxy gets obfuscated too, for example by using a domain generation algorithm to switch server names every so often, and by changing, or “fluxing”, DNS entries so that the browser proxies move around on the internet.
The bypass proxy then fetches the desired ad from the ad server, and rewrites its content so that any links to the real ad server that are embedded in the ad are themselves adjusted to go via the proxy.
This rewriting isn’t strictly necessary, because each recursive request to the ad server would go back through the bypass loader, get blocked and thus detected, and then get rewritten to go via the proxy anyway. But rewriting the links inside each ad makes things much faster, because only the first visit to the ad server needs to go through the test-to-see-if-it-will-be-blocked process.
If this sounds like how cybercrooks fight back against security products, with obfuscated JavaScript, disguised URLs, andregularly changing proxy servers and DNS records, don’t be surprised.
The principle is generic: create an algorithm which examines data objects and if they are in set X, rewrites them so that they are in not-X.
WHAT HAPPENS NEXT?
We’re not sure how well this trick is going to work.
We’re not thinking about the technological aspects here, but the cultural ones.
If I’ve blocked your ads – whether I don’t like them, don’t trust them, or both – then blocking me from your site unless I agree to unblock those ads seems perfectly reasonable.
If we can reach a willing buyer/willing seller compromise, then we will both end up happy; if not, then neither of us will end up with something at the expense of the other.
But forcing ads on me, especially if I’ve blocked your ads because of of security concerns and yet you are tricking my browser into displaying them in a way I won’t notice until it’s too late…
…how is that going to win me over?
As a commenter pointed out when we last discussed malvertising:
No, we don’t expect companies to give us everything for free. But neither should companies expect us to sacrifice our safety for their product. It’s a risk evaluation.
Perhaps a better approach would be to set about building an ad network that people were willing to unblock out of choice?
Isn’t that better than giving them an ad network with which they end up playing a cat-and-mouse game of detect-evade-detect-evade?
