Windows_Security

Level 19
Content Creator
Verified
Joined
Mar 13, 2016
Messages
935
Operating System
Windows 7
#1
Raymon Hill has released a new extension which records which (third-party) websites you have engaged when surfing the internet. After installation when you click the UBO-Scope ICON it informs you about the third-party exposure.

When you click the icon, you get the default tile unfiltered (I guess alphabetic) view. When you click on the corner left, you can switch from tile to list view. Clicking on the filter icon on the right sorts the websites on exposure percentage (from high to low).



In the options screen you change the number of days UBO-Scope records third-part exposure. You can also export the list to build your own (anti)tracking lists for UBlock, AdGuard or AdBlock..

These numbers are illustrative when disabling advertisement and tracking protection and clicking on a few major news websites :sick:.

upload_2017-12-2_9-33-54.png


IMPORTANT NOTICE:
UBO-Scope shows EXPOSURE not TRACKING. Many major websites have committed themselves to the self regulation FAIR USE/NO-TRACKING policy. Problem is when you bother to read the details of the self regulating policies of the advertising business, your definition of FAIR USE/NO TRACKING may differ from their FAIR USE/NO TRACKING definition.

POSSIBLE USE: When you try out new filterlists you can get an idea of the effectiveness of this new filterlist. The reverse applies also,when you remove a filterlist, you can see how much the exposure increases. It would be handy when the optios screen had a RESET button. When you click on a website it shows detailed info in the header. It also informs you about the nature of that website (e..g. Advertising or Content).

UBO-Scope can be used to personalize your blocklists. This extension explains why some well maintained blocklists are so effective (Adguard basic DNS, Disconnect simple ad filter). It costs a lot of money to set up an ad & tracking service with a large infrastructure. In most people's personal webspace, it is very unlikely you will encounter more than 5000 different ad- and tracking services in your entire living life (there is simply no market for more companies offering these services, see for instance Trackers | Better).

For blocking malware it is better to play it by the numbers and stack blocklists of market leaders with a lot of traffic e.g. use Norton DNS, Microsoft smartscreen/Google safe surfing + extension of a large Anti-virus (AVAST or Bitdefender). No community filter list has the means to outperform those number crunchers (in terms of webtraffic and automated web crawlers to collect the input).

When blocking only 30 known ad&tracking websites with a simple Javascript blocker (Sybu), surfing the same 6 news websites, clearly shows that Google Tag Services is the only third part which is used on more websites (red) All all others are just on just one website (17 = 17% = 1/6).

upload_2017-12-2_10-34-19.png
 
Last edited:

Windows_Security

Level 19
Content Creator
Verified
Joined
Mar 13, 2016
Messages
935
Operating System
Windows 7
#3
I found it just recently. I annoyed the developer of uBlock0rigin (Raymond Hill) with my critique that it made little sense to enable so many filter lists by default. Most of the community filter lists contain a lot of dead wood (obsolete old entries). Better stick to a few well maintained filterlist backed by commercial companies (Disconnect, Adguard, MalwareBytes). So mr. Hill decided to write an extension to proof my wrong (and put me on his ignore list).:)

For the record: I don't have anything personal against Raymond Hill (otherwise I would not compliment him with this nice add-on). I also need to tell you that I am one of the bad guys (in online marketing & sales). People tend to forget that advertising helps to keep the internet free. It is like love and marriage: you can't have one without the other.

Wel done and thank you Mr Hill (y)
 
Last edited:

TerrakionSmash

Level 18
Verified
Joined
Nov 17, 2016
Messages
855
Operating System
Other OS
Antivirus
Sophos
#4
What exactly are 'the too many enabled' lists? Easylist+EasyPrivacy+Peter Lowe's seems usable and well maintained and with little overlap and most people would probably like Easylist for the cosmetic filtering and blocking the annoying ads and popups on more fishy/overzealous sites.
The other enabled lists are malware lists which this extension doesn't measure. uBlock filters are purely supplementary to those 3.

This deadwood problem seems to be the type of problem Adguard's optimized filters is trying to remedy.
 
P

plat1098

Guest
#5
I currently use it for Google Chrome, the numbers are nice and low for the most part, don't need to fiddle with uBlock Origin for the moment. It's easy to think it's an blocker enhancement or blocker itself. It is not, it is a monitoring tool. Only have a very basic understanding of it so I kinda appreciate this posting. (y)

uBOscope.jpg
 

Windows_Security

Level 19
Content Creator
Verified
Joined
Mar 13, 2016
Messages
935
Operating System
Windows 7
#6
These are facts, not something I make up or theorize about.

Ad & Tracking blocklists
I can put adds on my website, but in order to get paid for the clicks per view I have to proof that those views were offered to people falling into the targetted market segments of the company paying for the advertisement. Because the infrastructure behind this costs a lot of money, there are only a limited number of companies offering these services. So there are limitations on supply side of the business case for a new ad & tracking service. Since there are already a lot of the ad & tracking services, there is also a limitation on how much companies can earn enough income to make a living out of it.. The market for online ads has a ceiling like any other market in the world has a maximum number of customers creating demand in it. So there is also a limitation on the demand side. As said you will have a hard time engaging with more than 5000 different ad & tracking companies in your entire life. So any blocklist with over 20.000 entries must contain a lot of dead wood. Better stick to a few well maintained filterlist backed by commercial companies (Disconnect, Adguard, MalwareBytes).

Malware blocklists

For blocking malware URLs, you need to collect malware URL;s in the first place. So players with a huge worldwide infrastructure (like backbone DNS, browsers/search engines and AntiVirus solutions with a lot of users) are the ones who have the BIGGER IS BETTER advantage. They also have the infrastruture to analyse webtraffic (with machine learning) and automate URL collection (with webcrawlers). So it is better to play it by the numbers and stack blocklists of market leaders with a lot of traffic e.g. use Norton DNS, Microsoft smartscreen/Google safe surfing + extension of a large Anti-virus (AVAST or Bitdefender). No community filter list has the means nor the money to come close to those number crunchers (less users, less webtraffic, les advanced automated analysis, etc)

Note: Lazy Chrome users can have both with Avast Online Security. Just set trackers to block automatically (this also includes a fair share of the advertising services). Add Proper blocker to deal with javascript pop-ups and overlays and your are done.
 
Last edited:

Windows_Security

Level 19
Content Creator
Verified
Joined
Mar 13, 2016
Messages
935
Operating System
Windows 7
#7
I currently use it for Google Chrome, the numbers are nice and low for the most part, don't need to fiddle with uBlock Origin for the moment. It's easy to think it's an blocker enhancement or blocker itself. It is not, it is a monitoring tool. Only have a very basic understanding of it so I kinda appreciate this posting. (y)

View attachment 175075
Correct it is only a monitoring tool

Chrome with uBlock0rigin on my 2 GB Asus Transformer. uB0 rules are determined with Ub0-Scope. With so few rules (70) uB0 uses little memory..

upload_2017-12-6_7-20-5.png
 

Windows_Security

Level 19
Content Creator
Verified
Joined
Mar 13, 2016
Messages
935
Operating System
Windows 7
#9
Choose options, then you see it (beware it is still a bit buggy, need to try a few times before it actually exports). After a few days I disabled UB0-Scope.

upload_2017-12-6_7-47-43.png


Note when I browse non-bookmarked websites, I always enable tthird-party blocking in uB0.
 
Last edited:

Windows_Security

Level 19
Content Creator
Verified
Joined
Mar 13, 2016
Messages
935
Operating System
Windows 7
#12
I thought you were the type of guy who chooses usability first.
You are correct on my Transformer (W10) I use Edge with Adguard and on my Desktop (W7) I use Firefox with internal blocker. I use Chrome for work, so browse a few websites for news. When I have to write content and am searching for facts/data, I put UBO in 3p-blocking mode. I don't use Ub0 anymore for personal surfing.

This deadwood problem seems to be the type of problem Adguard's optimized filters is trying to remedy.
I also tested some sites with different filter lists. Looking only to the number of connected domains uBo-scope helps to proof that less is more ;). The total number of connected domains reflects the effectiveness of the filters used as a ballpark figure

upload_2017-12-29_17-12-45.png


So TerrakionSmash is right Adguard base + spyware + Peter Lowe's seems to be as effective as the default rules of uBlock origin (with a lot of deadwood of old non active domains)
 

TerrakionSmash

Level 18
Verified
Joined
Nov 17, 2016
Messages
855
Operating System
Other OS
Antivirus
Sophos
#13
I also tested some sites with different filter lists. Looking only to the number of connected domains uBo-scope helps to proof that less is more ;) looking to the domains connected, which reflects filter effectiveness as ballpark figure

View attachment 177132

Adguard base + spyware + Peter Lowe's versus default rules of uBlock origin, besided checking trackers
But news sites would be the first to have filter lists, won't they, given that they're both popular and notoriously have additional unneccessary scripts.
 
Likes: AtlBo

Windows_Security

Level 19
Content Creator
Verified
Joined
Mar 13, 2016
Messages
935
Operating System
Windows 7
#14
@TerrakionSmash

What you say is true, most Alexa top 500 websites are well covered with Peter's Low list only.

But there is a reason why the adguard+peterlow list work just as well as all the default lists for most people (certainly for people living North-America and Europe).

There are not an unlimited number of CAR-makers, there are not an unlimited number of SUPERMARKETS, there are not an unlimited number of CONTENT AND DELIVERY servers and there are not an unlimited number of ADSERVING companies.

The reason for any market to have a limited number of companies offering a product or a service is:
  1. The market (number of customers) is limited, so the DEMAND side is limited
  2. To start up an adserving company you need a infrastructure which costs a lot of money. Money is a threshold for new competitors to enter a market, so the SUPPLY side is also limited
Only when new technology is available, the existing market balance will be disrupted.

That is a law of business, so all those filterlist with so many dead domains have very little additional value.
 
Last edited:

Slyguy

Level 39
Content Creator
Verified
Joined
Jan 27, 2017
Messages
2,856
Operating System
Other OS
#15
I found it just recently. I annoyed the developer of uBlock0rigin (Raymond Hill) with my critique that it made little sense to enable so many filter lists by default.
But you are correct in stating this. Which is why I disable almost all of his default lists, most are irrelevant and simply add to the weight of it and potential latency.
 
Joined
May 16, 2018
Messages
261
Operating System
Windows 10
Antivirus
Cylance
#17
This was a good discussion. It should not have ended... I still have not learned enough.

The squidblack lists seem to be maintained...

Ok, continue on.
 

Windows_Security

Level 19
Content Creator
Verified
Joined
Mar 13, 2016
Messages
935
Operating System
Windows 7
#18
@Burrito and @oldschool

Here is the export of my Filter I use with NanoAdblocker on Edge (of Asus Transformer I use for travel).. It contains around 2000 domains from DNT simple Ad-filter and (the Apple app) trackers from Better-FYI. I only contains COM, NET, ORG, INF domains because I started using it along side uBO-companion on Chrome (and I block all javascript on Chrome with exception rules, allowing high level country domains, [*.]com, [*.]inf, {*.]org and [*.] net.

Easy way to work: copy dark RED dains from uBO-companion to uBO my rules RED_DOMAIN * * BLOCK, when it does not break anything, add it amonth later to My Filters as ||RED_DOMAIN^ and remove it from my rules).

I only use My Filters (don't need more with my surfing habits). This covers ads and trackers from Alexa Top100 news sites and Top500 websites.

I run uBlock with third-party blocking. When I do my morning read on news sites or visit websites from my bookmarks, I just disable 3rd-party blocking. The 2K rules will do (you could add the easylist for your country to it).

Let's hope Raymond Hill ads an export function to uBO-companion, so you can import the list in My Flters of uBlock. uBO is still beta, so GorHill asked to not post feature requests on Github
 

Attachments

Last edited:

TerrakionSmash

Level 18
Verified
Joined
Nov 17, 2016
Messages
855
Operating System
Other OS
Antivirus
Sophos
#19
Here is the export of my Filter I use with NanoAdblocker on Edge (of Asus Transformer I use for travel).. It contains around 2000 domains from DNT simple Ad-filter and (the Apple app) trackers from Better-FYI. I only contains COM, NET, ORG, INF domains because I started using it along side uBO-companion on Chrome (and I block all javascript on Chrome with exception rules, allowing high level country domains, [*.]com, [*.]inf, {*.]org and [*.] net.
Woah! You're really pushing it. I wouldn't go to the hassle of setting this up especially because at some point, you would think the law of diminishing marginal returns would apply compared to the cost of setting it up.
I only use My Filters (don't need more with my surfing habits). This covers ads and trackers from Alexa Top100 news sites and Top500 websites.
Have no need of updates then?
 

Windows_Security

Level 19
Content Creator
Verified
Joined
Mar 13, 2016
Messages
935
Operating System
Windows 7
#20
No need for updates, that is where ulock companion comes in. As said I just combined DNT add filter and Better FYI trackers into an excel list and only converted com, net, inf, org into uMatrix rules. Umatrix drowns all invalid rules frim other country domains (still in ad block format). Then copy cleaned list back to notepad and change back uMatrix rules to ad block rules with 5 change alls. Seems like a lot of work, but takes less than 5 minutes to complete. After that I have the most compact list covering Alexa 500 and top 500 news / media websites of the world.