Q&A AdGuard tweaking (Yes, I confess, I did it again, changed adblocking strategy)

Lenny_Fox

Level 22
Verified
Oct 1, 2019
1,125
Yes, I admit it is an addiction trying to get the most out of the combo DNS - Browser - Extensions, but to look at the bright side of life and put this in perspective, this is a decent addiction, does not cost me any money and does not offend people nor law.

I tend to switch between uBlockOrigin and Adguard extension and vica versa. I was happily using the combo NextDNS-uBlockOrigin until @oldschool and @silversurfer started to post about ClearURL and NeatURL. This triggered me to have a look at AdGuard Extension again. Benefit of Adguard is that it also has an option to check for malware (I have Smartscreen disabled because it sends full URL) PLUS AdGuard extension has a stealth mode (which is disabled by default). This stealth mode also has an URL cleaning option. The default set of URL parameters can be easily tweaked, by simply adding comma seperated parameters to the text input box.


Cleaning URL's
I added some tracking parameters of popular tracking networks in AdGuard's stealth option:

utm_source,utm_medium,utm_term,utm_campaign,utm_content,utm_name,utm_cid,utm_reader,utm_viz_id,utm_pubreferrer,utm_swu,utm_referrer,utm_social,utm_social-type,utm_place,utm_userid,utm_channel,gs_l,gs_Lcp,client,sclient,ei,sei,gws_rd,gs_gbg,gs_mss,gs_rn,ved,uact,fbclid,_hsenc,_hsmi,__hssc,__hstc,hsCtaTracking,sc_campaign,sc_channel,sc_content,sc_medium,sc_outcome,sc_geo,sc_country,elqTrackId,elqTrack,assetType,assetId,recipientId,campaignId,siteId,s_cid,itm_source,itm_medium,itm_campaign,itm_content,itm_term,spReportId,spJobID,spUserID,spMailingID,hmb_campaign,hmb_medium,hmb_source,mkt_tok,mc_cid,mc_eid,pk_campaign,pk_kwd,pk_source,pk_medium,pk_content

EDIT: it is easier to add the parameters Security123 made out of this list above in the next post!

__________
 
Last edited:

HarborFront

Level 59
Verified
Content Creator
Oct 9, 2016
4,835
Yes think is in My filters add.
Hi

I added to Adguard for desktop the followings. However, in the screenshot below what's added are in green color which are different from what I added those abused TLDs previously which are in white color. Anything wrong here? And I add back ei in "URL tracking parameters to remove"?

$removeparam=ei
@@||youtube.com^$removeparam=ei
@@||googlevideo.com^$removeparam=ei

1610371480847.png
 

HarborFront

Level 59
Verified
Content Creator
Oct 9, 2016
4,835
When is add one by one parameter with $removeparam= then block tracking parameter is ok, but if all parameters in one line is not working maybe I wrong
Also add this in user filters

Code:
$removeparam=gs_lcp
$removeparam=ga_source
$removeparam=uact
$removeparam=sclient

link for now is this clean
https://www.google.com/search?hl=en&gl=us&q=
When I click your link I got this

httpx://www.google.com/webhp?gl=us


But when I go to www.google.com it only display, without adding your

$removeparam=gs_lcp
$removeparam=ga_source
$removeparam=uact
$removeparam=sclien

httpx://www.google.com/


Mine is cleaner
 

petok

Level 1
Sep 19, 2011
32
I use long time this link from google set to English, no locale language
The hl parameter specifies the interface language (host language) of your user interface
The gl parameter value is a two-letter country code.
The gws_rd is google web services redirect and add to ssl encryption

httpx://www.google.com/?hl=en&gl=us&gws_rd=ssl
 
Last edited:

Tiamati

Level 11
Verified
Nov 8, 2016
502
According to my post #27 above fb is taken from Trace extension. You got comment on fb?
I didn't notice the FB. I thought it was wrong. Any particular reason to avoid using the other 2 i mentioned?

fb_action_ids
fb_action_types


Isn't available at Edge store so you need allowing Google ones which opens another security hole.
In that case i would pick Emsisoft Browser Security (for security, performance and privacy).

Or any of these>
Netcraft Extension
Norton Safe Web
Avira Safe Shopping
 

HarborFront

Level 59
Verified
Content Creator
Oct 9, 2016
4,835

Lenny_Fox

Level 22
Verified
Oct 1, 2019
1,125
Since 1.32 uBlockOrigin also has the ability to remove URL-parameters. GorHill is planning to roll it in one of the upcoming releases.

When you want to try it out just enable "FilterAutherMode"
1610649502936.png

EXAMPLE: stripping google search parameters with new QUERYPRUNE option (link uBO wiki)
||google.*^$queryprune=|ei|gs_lcp|psi|psy|uact|ved,domain=google.*

RESULT a clean search URL passing little to no info when you click on a link
1610649994465.png


PS: YEs I confess, I did it again hopping from AdGuard to uBlockOrigin this time ;)
 
Last edited:

HarborFront

Level 59
Verified
Content Creator
Oct 9, 2016
4,835
Since 1.32 uBlockOrigin also has the ability to remove URL-parameters. GorHill is planning to roll it in one of the upcoming releases.

When you want to try it out just enable "FilterAutherMode"

EXAMPLE: stripping google search parameters with new QUERYPRUNE option (link uBO wiki)
||google.*^$queryprune=|ei|gs_lcp|psi|psy|uact|ved,domain=google.*

RESULT a clean search URL passing little to no info when you click on a link
View attachment 252960

PS: YEs I confess, I did it again hopping from AdGuard to uBlockOrigin this time ;)
Does it come with a set of pre-blocked URL parameters to be removed just like in Neat URL? If not you'll need to craft the queryprune filters yourself

As quoted from the uBO wiki link

Poorly crafted queryprune filters can have deleterious effects on performance, experienced filter authors are expected to understand well how to craft optimal filters.
 
  • Like
Reactions: Protomartyr

rain2reign

Level 6
Jun 21, 2020
276
Since 1.32 uBlockOrigin also has the ability to remove URL-parameters. GorHill is planning to roll it in one of the upcoming releases.

When you want to try it out just enable "FilterAutherMode"

EXAMPLE: stripping google search parameters with new QUERYPRUNE option (link uBO wiki)
||google.*^$queryprune=|ei|gs_lcp|psi|psy|uact|ved,domain=google.*

RESULT a clean search URL passing little to no info when you click on a link
View attachment 252960

PS: YEs I confess, I did it again hopping from AdGuard to uBlockOrigin this time ;)
In regards to the scriptlets question, i do not know as i never used that line of code. BUT if it's just for AdGuard maintained filterlists. You can find the uBlock optimized filterlists maintained by the AdGuard team themselves.

Example: https://filters.adtidy.org/extension/ublock/filters/3.txt replace the ublock part in the url with chromium for the AdGuard compatible list or the other way around for the uBlock compatible list.
 
  • Like
Reactions: Protomartyr

Lenny_Fox

Level 22
Verified
Oct 1, 2019
1,125
Does it come with a set of pre-blocked URL parameters to be removed just like in Neat URL? If not you'll need to craft the queryprune filters yourself

As quoted from the uBO wiki link

Poorly crafted queryprune filters can have deleterious effects on performance, experienced filter authors are expected to understand well how to craft optimal filters.
1. No as far as I know you have to construct them yourself. As with many other advanced uBO features, in time they will appear in the uBO build-in lists.

2. Good point better to change the rule for AG compatibility (thx @petok uBO also recognises AG's removeparam]) and reduced performance overhead:

||google.*^$removeparam=ei|gs_lcp|oq|sclient|source|uact|ved|psi|psy,domain=google.*
 
Last edited:

petok

Level 1
Sep 19, 2011
32
I update just add others parameter and others just need test 4 new parameters to see what is loading then I will add to rule.
I see google have tricks in loading pages for now this is update thanks for other member for parameter

Code:
||google.*^$removeparam=aq|ei|gs_l|gs_lcp|oq|pq|psi|psy|sa|sclient|source|sourceid|uact|ved

@Lenny_Fox why is add domain=google.* on end ?
 
Last edited:
  • Like
Reactions: Protomartyr

Lenny_Fox

Level 22
Verified
Oct 1, 2019
1,125
That is for uBlockOrigin. It applies the rule only on domains when visiting a website with google as domain name for any country code (* is wildcard) e.g. on google.com, google.fr, google.de etc.

This way the rule is only processed when seaching with google
 

SeriousHoax

Level 38
Verified
Mar 16, 2019
2,717

Lenny_Fox

Level 22
Verified
Oct 1, 2019
1,125
I thank @SeriousHoax for attending me on Yuki's comments. Fun-fact when I read Yuki's earlier comments about HTTP third-party rule I did not understand what he meant, now he worded it differently i think I understand what he is advising. Same with $queryprune (now $removeparam), first time I followed the documentation at that time (see note 1), obviously something was depreciated according to current documentation (see note 2).

It would be more helpful to write a better manual than complaining about wrong usage and people being to dumb to use advanced rules features. Just giving it a second go and hopefully his Yuki-ness will comment on the rules below for educational purpose.
______________________________________________________________________
||google.nl/search?*aq$removeparam=aq,domain=google.nl
||google.nl/search?*ei$removeparam=ei,domain=google.nl
||google.nl/search?*gs_l$removeparam=gs_l,domain=google.nl
||google.nl/search?*gs_lcp$removeparam=gs_lcp,domain=google.nl
||google.nl/search?*oq$removeparam=oq,domain=google.nl
||google.nl/search?*pq$removeparam=pq,domain=google.nl
||google.nl/search?*psi$removeparam=psi,domain=google.nl
||google.nl/search?*psy$removeparam=psy,domain=google.nl
||google.nl/search?*sa$removeparam=sa,domain=google.nl
||google.nl/search?*sclient$removeparam=sclient,domain=google.nl
||google.nl/search?*source$removeparam=source,domain=google.nl
||google.nl/search?*sourceid$removeparam=sourceid,domain=google.nl
||google.nl/search?*uact$removeparam=uact,domain=google.nl
||google.nl/search?*ved$removeparam=ved,domain=google.nl
_______________________________________________________________________
In stead of writing one general purpose rule, I made it specific to google.nl, I also split the different parameters into individual rules so a rule has more tokens and they can be put in a token-bucket for more efficient processing.

Note1: I added a picture of an actual rule of uBo build-in filters (using the pipe) to show using pipe (|) in version one was according the uBO wiki-documentation at that time
Note2: I added a picture of current documentation. so everyone can check the version two rules are according the documentation available at the moment of writing this post
 

Attachments

  • 1610822169819.png
    1610822169819.png
    5.5 KB · Views: 90
  • 1610822273138.png
    1610822273138.png
    77.5 KB · Views: 94
Last edited:

Lenny_Fox

Level 22
Verified
Oct 1, 2019
1,125
Follow up post, actual load time testing of removeparam

I compared this with the multi-rule version with single rule version. The multi-rule versions were written in exactly the same manner as the queryprune example I found in the uBlock build-in list. Assuming the author of the build-in uBO filter knows how to write efficient rules, the copied multi-rule version should also be efficient.

When using uBO's build in logger I could not notice any performance differences (I repoeated the test 5 times)

Single ineffecient version removeparam
1610872687046.png


Muti-rule removeparam version following structure and syntax of queryprune rule of uBO build-in filter
1610872765381.png


So using the uBO logger ihe inefficient rule performed the same as the efficient rule. So I decided to use another tool and to give credit to Yuki the multi-rule version was more efficient (4 digit after the decimal break). So a single in efficient line will cost 0.0001 to 0.0002 response time. Which is a lot when uBo easily processes half a million rules in half a second.

Lesson learned: we need crafted rule writers in this forum like Yuki :)

EDIT this evening I researched what info these parameters contained, so only kept two removeparam's and dumped all others (I am not paranoid, just playing with stuff :) so it is time to lower my level of excitement on the new found feature and only use it when it really increases privacy). A

! strip google click identifier
||google.nl/*glcid$removeparam=glid,domain=google.nl
! strip google search location parameter
||google.nl/*gs_lcp$removeparam=gs_lcp,domain=google.nl

From a performanc epoint of view these rules worked as good (4 digits behind decimal point) or other factors influencing response time had more impact (other network traffic and/or available bandwith) to notice the difference.
 
Last edited:

HarborFront

Level 59
Verified
Content Creator
Oct 9, 2016
4,835
Sorry I made a typing mistake correct rules are

! strip google click identifier
||google.nl/*gclid$removeparam=gclid,domain=google.nl
! strip google search location
||google.nl/*gs_lcp$removeparam=gs_lcp,domain=google.nl
I'm not familiar with what you are doing. However, using one example

||google.nl/*gs_lcp$removeparam=gs_lcp,domain=google.nl

I can see that the google.nl (front and back) refers to searches in Netherlands. Am I right?

But if I want to search US, UK and other countries can use a wild card like google.* instead of a specific country? Google searches are used if you use Google as the search engine, right? If I'm using say DDG, Qwant etc then no need of them, correct?

Thanks
 
  • Like
Reactions: Protomartyr

rain2reign

Level 6
Jun 21, 2020
276
I'm not familiar with what you are doing. However, using one example

||google.nl/*gs_lcp$removeparam=gs_lcp,domain=google.nl

I can see that the google.nl (front and back) refers to searches in Netherlands. Am I right?

But if I want to search US, UK and other countries can use a wild card like google.* instead of a specific country? Google searches are used if you use Google as the search engine, right? If I'm using say DDG, Qwant etc then no need of them, correct?

Thanks
Whether you use ||google.[<TLD-here>] or ||google.* doesn't matter in uBlock from what I know, but if you exclusively visit a particular TLD variant of Google than it might be more efficient for you to fill in that one, no? I personally have always with go google.* unless I was looking to either block, hide or allow something very specific when it comes to Google.
 
Last edited:
Top