Updates How AdGuard scanned the entire web in search of hidden trackers

Gandalf_The_Grey

Level 48
Verified
Trusted
Content Creator
Apr 24, 2016
3,793
As content blocking has become widespread, most tools for excessive tracking proved to be fairly useless. But with the market moving more and more towards massive data collection, the tendency was to push it as far as possible. Some opt for a blatant approach, and some seek more inventive ways to collect users' data.

One of such more subtle methods involves CNAME. A CNAME record, which is short for 'Canonical Name record', is a type of DNS record that maps one domain name (an alias) to another (the canonical name), instead of mapping this domain directly to an IP address. It's a basic function used by millions of websites to create unique subdomains for different services, such as mail, search, etc. To allow for seamless interaction, the subdomains are trusted just like the primary domain.

CNAME-cloaked tracking abuses this fundamental mechanic and creates many more problems than just unwelcome data collection.

By using a CNAME record, an external tracking server can be disguised as a subdomain of a website the browser trusts, and the tracking cookies will be accepted as "first-party" ones. What's worse, it works the other way around too, and the cookies meant for the primary domain may be shared with the tracker-in-disguise. The third party can receive all kinds of data, from the user's name and contact details to authentication cookies used to identify their session and to keep them logged onto the website.

According to a recent research paper by Yana Dimova, Gunes Acar, Wouter Joosen, Tom Van Goethem, and Lukasz Olejnik, cookie leaks occur on 95% of the websites that employ such trackers. The research emphasizes that CNAME-cloaked tracking fools the basic web security tools and may lead to major security and privacy breaches.

Browsers themselves can't protect users from CNAME-cloaked tracking. But content blockers can: AdGuard and AdGuard DNS, as well as uBO on Mozilla Firefox already block such "hidden trackers". Still, due to limitations in Chrome, Chromium and Safari, regular extensions can't dynamically resolve hostnames and remove trackers. They're limited to filter lists, and it's hard to imagine someone would check the whole web in search for CNAME-cloaked trackers to compile a 'perfect' comprehensive filter list.

Wait, actually, we did just that. Thanks to our own DNS server, plus a set of standalone and browser-based content blocking tools, we've been able to hunt the hunters (or rather track the trackers), list them, and block them. Now we're making the full list of all known CNAME-cloaked trackers publicly available as a part of the AdGuard Tracking Protection Filter. We've also published it on GitHub so that other content blockers could use it. This is the most complete auto-updating repository of actively used hidden trackers by now, consisting of more than 6000 entries. The list is to be updated on a regular basis to add new hidden trackers as they're being detected.

Does this mean CNAME-cloaked tracking is dealt with once and for all? Unfortunately not. We plan to keep the filter list up to date, but the number of hidden trackers constantly grows, meaning that the number of blocking rules will be increasing as well. The problem is, Safari and Chrome in their chase after the total control over content blocking limit the number of blocking rules to 50,000 and 150,000 (as planned in Manifest V3) respectively. Even today we see that Safari's 50,000 rules are barely enough to protect yourself against ads, trackers, and everything else bad that's lurking on the web. One day they will simply run out of space to protect users against actual threats, and this day is closer than you might think.
 

SeriousHoax

Level 36
Verified
Mar 16, 2019
2,599
Funny thing is, Adguard DNS was actually blocking SmartScreen a few days ago for me through CNAME cloaking. It was fixed for me almost a full 24 hours later. Anyway, there's always gonna be odd false positives here and there. It's good to see that they are working hard to track these CNAME trackers and making it available for everybody. NextDNS has both Adguard DNS and Adguard Tracking protection filters, so that is great (y)
 

SecurityNightmares

Level 40
Verified
Jan 9, 2020
2,946
This is already handled by nextdns/cname-cloaking-blocklist, and appears as a specific and separate feature in NextDNS.

The reason they advertise the 6000+ is that I'm assuming their system is not able to block at the CNAME level, so they do something less ideal of scraping the web to try to list all QNAMEs pointing to those few CNAMEs, and try to keep this list as complete and up-to-date as they can.

So, using NextDNS is better (y)
 

blackice

Level 33
Verified
Apr 1, 2019
2,205

blackice

Level 33
Verified
Apr 1, 2019
2,205
Funny thing is, Adguard DNS was actually blocking SmartScreen a few days ago for me through CNAME cloaking. It was fixed for me almost a full 24 hours later. Anyway, there's always gonna be odd false positives here and there. It's good to see that they are working hard to track these CNAME trackers and making it available for everybody. NextDNS has both Adguard DNS and Adguard Tracking protection filters, so that is great (y)
How did you know Adguard was blocking smartscreen?
 

SeriousHoax

Level 36
Verified
Mar 16, 2019
2,599
How did you know Adguard was blocking smartscreen?
From NextDNS log. I ran an app that I downloaded and it was showing that SmartScreen can't be reached. It was being blocked by Adguard DNS filter according to the NextDNS log. Then I changed from NextDNS to Adguard DNS, flushed dns, restarted the system and it was being blocked once again. Then switched to Cloudflare and SmartScreen was working again. So surely it was Adguard DNS. It was fixed the next day.
 

SecurityNightmares

Level 40
Verified
Jan 9, 2020
2,946
From NextDNS log. I ran an app that I downloaded and it was showing that SmartScreen can't be reached. It was being blocked by Adguard DNS filter according to the NextDNS log. Then I changed from NextDNS to Adguard DNS, flushed dns, restarted the system and it was being blocked once again. Then switched to Cloudflare and SmartScreen was working again. So surely it was Adguard DNS. It was fixed the next day.
Not the first false positive with AdGuard but to be fair NextDNS have some too ("thanks" to Steven Black's list).
That's why i recommend (only) OISD filter list in my NextDNS guide
 

ErzCrz

Level 9
Verified
Aug 19, 2019
448
Adguard extension is great though I get slow loading of some pages. I'm still persevering with it.

With uBO I added @Lenny_Fox 's filter to block Cname trackers not in EasyList.

! Block CNAME trackers (not covered in EasyList Privacy)
/id?d_visid_ver=$~xmlhttprequest
/id?d_visid_
/b/ss/*&age=
!

Currently using Firefox testing out other configurations.
 

Gandalf_The_Grey

Level 48
Verified
Trusted
Content Creator
Apr 24, 2016
3,793
Adguard extension is great though I get slow loading of some pages. I'm still persevering with it.

With uBO I added @Lenny_Fox 's filter to block Cname trackers not in EasyList.

! Block CNAME trackers (not covered in EasyList Privacy)
/id?d_visid_ver=$~xmlhttprequest
/id?d_visid_
/b/ss/*&age=
!

Currently using Firefox testing out other configurations.
You can speed up the page loading with the AdGuard extension by disabling in the general settings "Phishing and malware protection" (Safe Browsing or SmartScreen are much better) and while you are there "Activate the most appropriate filters automatically" so that you don't end up with a lot of 'useless" language filters because you visited a site in a foreign language. In Miscellaneous select "Use optimized filters". With those settings AdGuard should be (almost) as fast as uBlock Origin.
 

ErzCrz

Level 9
Verified
Aug 19, 2019
448
You can speed up the page loading with the AdGuard extension by disabling in the general settings "Phishing and malware protection" (Safe Browsing or SmartScreen are much better) and while you are there "Activate the most appropriate filters automatically" so that you don't end up with a lot of 'useless" language filters because you visited a site in a foreign language. In Miscellaneous select "Use optimized filters". With those settings AdGuard should be (almost) as fast as uBlock Origin.
Thanks, I'll give that a try :)
 
Top