Adblocking innovation

Lenny_Fox

Level 22
Thread author
Verified
Top Poster
Well-known
Oct 1, 2019
1,120
Disclaimer: when you believe in large blocklists with 100.000 rules or more, stop reading this post (because this post is based on research data and not alternative facts and popular make believe :) ) Adblocking methods are developing into new interesting directions using several layers to increase effectiveness of ad-blocking and reduce website breakage due to bad filters.

What is wrong with old (large) block list approach?
  1. Large community based blocklists (like Easylist) contain a lot of dead rules
    A blog post based on a field research explains the scope of dead or stale rules in Easy Privacy blocklist. The Mounting Cost of Stale Ad Blocking Rules | Brave Browser . The problem with uer maintained black lists is that there is no automated mechanism to check whether that rule actually blocks something. That is why some adblockers like Adguard measure how many times rules are triggered and offer their own optimized version of the community based black lists. Also the build-in adblockers of Brave and Opera clean up their versions of the Easylist blocklist on a regular base.

  2. Large community based blocklist chase the wrong target.
    The problem with most community maintained blocklist is that they chase the symptoms and not the cause. The community chases ads on websites they surf to/on. But the ads on these websites are displayed are pushed to these website by an ad&tracking network. As Peter Low explains on his website (quote) "The ad banners that you see all over the web are stored on servers. Stopping your computer communicating with another computer can be quite simple. So, if you have a list of the servers used for ad banners, it's easy to stop ad banners even getting to your browser." . So blocking ads on individual websites like most community maintained list do, is putting the horse behind the cart.
These two problems makes large block list not more effective than small block lists ((link to 2015 study - less third party request is better)
1583318845517.png



What new techniques are evolving?
  1. Small blocklist with isolation.
    Firefox uses the block lists of Disconnect with containers (to keep parts of your online life separated into containers that preserve your privacy). When you are logged in to Google or Facebook and visit websites, you disclose a lot more info. FF containers prevent this.

  2. Small blocklist with smart white listing
    Microsoft also uses the (small) block lists of Disconnect and reduces website breakage with user engagement score (white listing known trackers based on the users interaction with a specific website)

  3. Small block list with heuristics based blacklisting
    This is how Privacy Badger works (when a single third-party hosts tracks you on three separate sites, this ad and tracking server will be automatically disallowed). The information provided on Privacy badger's website is not actual anymore. Ghostery also uses a similar heuristics but more advanced approach as Privacy Badger. Ghostery evaluates URL-parameters. When an URL- parameter has more than three times the same values in the last two days it will be filtered (link) This is more granular than Privacy badger (three strikes is out on domain level) and causes less website breakage. Ghostery's blocking URL parameters is simular to AdGuard's stealthmode blocking UTM parameters. Ghostery's approach is more granular than AdGuard's system wide URL-parameters filtering. This Ghostery/Cliqz claim is backed by research (paper).

  4. Machine learning (AdGraph)
    On the upcoming privacy and security symposium (link) a ' new' approach to use Machine Learning for adblocking is presented (called AdGraph link). Why is 'new' put in quotes because this Proof Of Concept Chromium browser (source code on Github link) is based on a 2018 study (link) sponsored and co-created with Brave (link).

    AdGraph automatically and effectively blocks ads and trackers with 97.7% accuracy. AdGraph even has better recall than filter lists, as it blocks 16% more ads and trackers with 65% accuracy. The analysis also shows that AdGraph is fairly robust against adversarial obfuscation by publishers and advertisers that bypass filter lists.

    1583324981949.png
The graph identifies advertising and tracking resources in websites based on the HTML structure, JavaScript behavior, and network requests made during execution. The PoC chromium uses a modified Blink (renderer) and V8 (Javascript) engine to classify and attribute third-party call traces of ad network. Although AdGraph much better detects evasion techniques of advertising networks and loads pages faster than stock Chromium on 42% of pages, and faster than AdBlock Plus on 78% of pages, this is also the Achilles heel of this approach. When Brave would decide to use this innovation, it would not only have to maintain its own fork (program code variant) of the browser (Chromium), but also a fork of the renderer (Blink) and the Javascript Engine (V8). This is probably the reason Brave has not embraced this technology from this 2018 study they co-funded and co-created.​
Humble suggestion of a junior member to browser developers:
Develop an adblocker using a short blocklist (for third-party cookies, fingerprinting and scripts like Privacy Badger's initial JSON blacklist) with Edge's user engagement white listing mechanism and Ghostery's heuristic based URL parameter filtering to limit third-party exposure with an data collection OPT-IN.​
With the collected OPT-IN data the "no-UID safe set" cloud white list is optimized using Machine Learning technology :). No-IUD safe set attributes are excluded in Ghostery's local " same URL-parameter value" analysis, for example 1920x1080 and 720x1280 are common URL-parameter values, but they probably represent the screen size, so should be excluded as a possible unique identifier. When a user does not OPT-IN, this adblocker does not use the cloud "URL-parameter safe-set" (but only local version). At browser launch the local cookie-fingerprint-script blacklist and "safe-set URL-parameter" whitelist are updated.​
 
Last edited:
F

ForgottenSeer 85179

This is also the right way and it's right that nowadays Filterlist doesn't work how people think.
Even Daniel Micay (GrapheneOS Dev) say that.

The problem is also what the sites go to first-party scripts instead of 3th party, so these lists get worse and worse.

It's awesome to see that these guys build such a solution. Hope they released it for Chromium-Edge too.
 

Burrito

Level 24
Verified
Top Poster
Well-known
May 16, 2018
1,363
...this post is based on research data and not alternative facts and popular make believe :)


But Lenny.... alternative facts and make believe are so much more fun. ;)

---

Thanks for another really good and educational post about adblocking. As the nature of ads being served to us and the possible responses to that now are changing relatively rapidly -- your posts on this subject are a real help to this community to assist us in keeping up and learning.

(y)(y)
 

TairikuOkami

Level 37
Verified
Top Poster
Content Creator
Well-known
May 13, 2017
2,664
I stopped using Privacy Badger, because that approach just did not work for me. It was barely blocking anything. :cautious:
Filters were built to block the most/least popular webpages, so they will always work. There are custom filters for the rest.
From all adult filtering options I have tried, a filter list with 400.000 domains is the only one, that blocks the most relevant stuff.
 

SeriousHoax

Level 49
Verified
Top Poster
Well-known
Mar 16, 2019
3,863
Sometimes the annoying cookie notices, popups, sign up notice, newsletter notice, etc are worst than ads for a better browsing experience. Cosmetic filtering is important too to get rid of those empty ad placeholders. For these there's no better alternative to filter based adblocking extensions I think.
 

oldschool

Level 85
Verified
Top Poster
Well-known
Mar 29, 2018
7,673
Sometimes the annoying cookie notices, popups, sign up notice, newsletter notice, etc are worst than ads for a better browsing experience. Cosmetic filtering is important too to get rid of those empty ad placeholders. For these there's no better alternative to filter based adblocking extensions I think.

I generally agree but I have to say Brave does a pretty decent job at these.
 

Lenny_Fox

Level 22
Thread author
Verified
Top Poster
Well-known
Oct 1, 2019
1,120
Sometimes the annoying cookie notices, popups, sign up notice, newsletter notice, etc are worst than ads for a better browsing experience. Cosmetic filtering is important too to get rid of those empty ad placeholders. For these there's no better alternative to filter based adblocking extensions I think.
First: Using a large block list is 9 out of 10 timed the reason you run into an anti-adblock wall.

Secondly: not confirming cookie or privacy prompts, results in 9 out of 10 times in not showing rich content like videos etc.

Thirdly: Google has announced that its internal adblocker will block flow disturbing first party prompts (signup newsletters) soon. Because this does not involve third party advertising in 9 out 10 times cases, this does not hurt Google's advertising business (in case you were thinking Google suddenly changed its policy towards ads :) )

I generally agree that the "block/hide option" of large blocklist extensiom is most suited for these nuisances.
 

SeriousHoax

Level 49
Verified
Top Poster
Well-known
Mar 16, 2019
3,863
First: Using a large block list is 9 out of 10 timed the reason you run into an anti-adblock wall.
I haven't faced an anti-adblock warning in a while maybe because I'm also subscribed to nano filters.
Secondly: not confirming cookie or privacy prompts, results in 9 out of 10 times in not showing rich content like videos etc.
Interesting but haven't faced this issue either.
Thirdly: Google has announced that its internal adblocker will block flow disturbing first party prompts (signup newsletters) soon. Because this does not involve third party advertising in 9 out 10 times cases, this does not hurt Google's advertising business
That sounds nice and would be interesting to see how it performs. I'm a Firefox main though so I won't be benefited if they don't follow.
I'm not sure I understand what you mean by "block/hide option".
I think he meant the cosmetic filtering and other filter rules that hide unwanted elements like empty ad placeholders and other annoyances. Also adblocking extensions lets you manually hide unwanted elements. Now go this link for example. Visit it with EasyList filter enable then visit without and you'll see what EasyList did there: Videos in Play Store listings will play automatically starting in September
 

oldschool

Level 85
Verified
Top Poster
Well-known
Mar 29, 2018
7,673
I haven't faced an anti-adblock warning in a while maybe because I'm also subscribed to nano filters.

Interesting but haven't faced this issue either.

That sounds nice and would be interesting to see how it performs. I'm a Firefox main though so I won't be benefited if they don't follow.

I think he meant the cosmetic filtering and other filter rules that hide unwanted elements like empty ad placeholders and other annoyances. Also adblocking extensions lets you manually hide unwanted elements. Now go this link for example. Visit it with EasyList filter enable then visit without and you'll see what EasyList did there: Videos in Play Store listings will play automatically starting in September
Brave adblock has an option to block elements but doesn't have the total eelement blocking ability of µBO not always as easy as µBO e.g I'm not proficient at making custom filters to overcome Brave's deficiencies, like with some placeholders I had to open developer tools and "copy selector" to remove it.
 
Last edited:

SeriousHoax

Level 49
Verified
Top Poster
Well-known
Mar 16, 2019
3,863
Brave adblock has an option to block elements but doesn't have the total element blocking ability of µBO, e.g I'm not proficient at making custom filters to overcome Brave's deficiencies, like with some placeholders, etc.
I'm gonna have to check Brave shield again.
Btw, the browser Lenny mentioned in the post is interesting. No windows version yet though.
 

Lenny_Fox

Level 22
Thread author
Verified
Top Poster
Well-known
Oct 1, 2019
1,120
Fun facts Brave vs Cliqz:
- privacy law: US vs DE (EU GDPR is much more strict than US ECPA )
- own browser : Yes (Brave Chromium fork) -vs Yes (Cliqz Firefox fork)
- own advertising and reward model: Yes (Brave rewards are percentage of ad-income) vs Yes (Ghostery Awards are a discount on offerings)
- open source and opt-out: Yes & Yes vs Yes & Yes
- contribute to development of adblock innovation: Yes (link already provided) vs Yes (cliqz-oss/adblocker)
- bottem line: Brave is the undisputed darling of privacy aware forum members vs Cliqz is the cursed devil in disguise by members of this forum

Help me understand the difference :unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure::unsure:



P.S. I am not using Brave or Cliqz (browser nor its extension Ghostery)
 
Last edited:

Stopspying

Level 19
Verified
Top Poster
Well-known
Jan 21, 2018
814
I used Ghostery for a couple of years when it first came onto the scene, then it was bought by Evidon and I found safer alternatives. I liked it back then, is it as good(or better) now? I'm guessing that the association with Cliqz means the answer is positive.

@Lenny_Linux - a well put together and informative article, thanks.
 

Lenny_Fox

Level 22
Thread author
Verified
Top Poster
Well-known
Oct 1, 2019
1,120
I decided to give Ghostery a spin. Opting out of the data collection is possible (it is not that difficult to find in the user interface):

1583410833002.png


I upped the setting a bit. Enabled all categories to block except ESSENTIAL and AUDIO/VIDEO PLAYER (left) and Selected to ' Block new trackers added to Ghostery by default' (right). To receive promotions and (coupon) discounts some website require tagmanager to work, so allowing ESSENTIALS is an obvious choice. I ran into a 'adblock wall' (please disable your adblocker) when blocking AUDIO/VIDEO trackers that is the reason why I allowed this category also. This is just a quick tour, so your mileage may vary when you copy these settings.

1583411155382.png


Ghostery informs you how much trackers are blocked (on domain level) and modified (filter URL-parameters to anonymize ). When you choose 'Restrict Site' everything is blocked for the current domain, selecting 'Trust Site' allows all (disables blocking). So using Ghostery on default settings and selectively restricting websites is a good fall back option when the above 'upped' settings blocks to much or trigger to many anti-adblock prompts (below left is Trusted, right is Restricted).

1583413095409.png



Ghostery heuristics seem to work well. It did find to two in-video bumper ads on Dutch websites which are not in Easylist (English + Dutch) filters (the unknown trackers left and right in picture below). What really impressed me was the fact that the second one (on the right) was a first party advertisement. Websites NOS.NL and NU.NL visited with the 'upped' settings.

1583411799612.png


DISCLAIMER: this is not an Ghostery promotion, have already removed it again (using build-in Edge-chromium anti-tracking and build-in Opera adblock with Peter Low's list). Jut wanted to see whether heuristics really deliver what Ghostery promises (and it did).
 
Last edited:

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top