true Adblocking with chromium

ruth · 2010-05-24 20:13:18

Hi everybody,

i hacked together a true adblocking solution for chromium:
http://pastebin.com/Qdu7dHsE
as you know, all known adblockers for chromium are esentially just hiding ads, they do not actually prevent them from being loaded.
now, it seems that i managed to intercept net access within webkit ;-)
just put for example the easylist ablock for for adblock plus into
~/.config/chromium/patterns.ini
(for example: https://easylist-downloads.adblockplus. … ylist.txt)
and if you start chromium from the console, you can watch the adblocker work:

[ruth@RedQueen chromium_work]$ ./src/out/Release/chrome
filtered URL: http://oas.heise.de/RealMedia/ads/Creatives/OasDefault/mjx/mjx.2009-11-17.0.js using Pattern: ^.*/RealMedia/ads/.*$
filtered URL: http://oas.heise.de/RealMedia/ads/Creatives/OasDefault/mjx/mjx.2009-11-17.0.js using Pattern: ^.*/RealMedia/ads/.*$
filtered URL: http://heise.ivwbox.de/cgi-bin/ivw/CP/news_7tage;/newsticker/?r=&d=47295.014024712145 using Pattern: /cgi-bin/ivw/.*
filtered URL: http://heise.ivwbox.de/2004/01/survey.js using Pattern: ^.*ivwbox.de.*$
filtered URL: http://heise.ivwbox.de/2004/01/survey.js using Pattern: ^.*ivwbox.de.*$

* can parse adblock plus filterfiles - well partially ;-)
* this code is not optimized in any way
* it currently does not work, if the sandbox is enabled (sandboxed processes do not have filesystem access, that is needed for actually reading the pattern file *gg*, is doable)
* it does not implement whitelisting
* does not implement filtering text ads (needs access to the DOM, is doable)
* probably full of bugs ;-)

having said that, the code already speeds up chromium dramatically.

i could _really_ need help on this thing, so if anyone is interested, just send me a mail,* ->patches<-* and other stuff ;-)

atm, im not interested in bugreports and questions how to use this code ,-)

Cheers,
ruth
deinococcus.radiophilus@gmail.com

drcouzelis · 2010-05-24 21:19:54

Great! I am interested in this becoming more stable. Will you be creating a PKGBUILD file for the AUR?

I believe the Iron web browser has true ad blocking, but the last time I tried it I experienced some technical difficulties. (it wouldn't connect to any websites)

ruth · 2010-05-24 21:35:06

drcouzelis,
yes, i heard that iron has this feature too...
BUT:
http://thoughtyblog.wordpress.com/2009/ … mium-fork/
http://neugierig.org/software/chromium/ … /iron.html
oh, i didnt find sourcecode for iron, i simply do not trust iron, that simple ;-)

it wouldnt be too hard to include it into
http://aur.archlinux.org/packages.php?ID=29440
for example ;-)

Cheers,
ruth

SiC · 2010-05-24 21:49:11

ruth wrote:

drcouzelis,
yes, i heard that iron has this feature too...
BUT:
http://thoughtyblog.wordpress.com/2009/ … mium-fork/
http://neugierig.org/software/chromium/ … /iron.html
oh, i didnt find sourcecode for iron, i simply do not trust iron, that simple ;-)
it wouldnt be too hard to include it into
http://aur.archlinux.org/packages.php?ID=29440
for example ;-)
Cheers,
ruth

Sourcecode for iron can be found at the bottom of this page.

http://www.srware.net/en/software_srwar … wnload.php

ruth · 2010-05-24 22:41:21

yes,
downloaded this thing a few months ago.
old, old stuff, that is not even complete.
nothing really usable to start with developing. ;-)

anyways, im not interested in a chromium vs. iron discussion here ;-)

i already told you my opinion about iron.
from here:
http://neugierig.org/software/chromium/ … /iron.html

(1) if you don't trust Google to not do something sneaky, you probably shouldn't be running software made by Google, and
(2) why would you trust code from some random third party more?

for me, not releasing the source code for a privacy oriented piece of software is just sick, imho.
the sourcecode on rapidshare(!!!) is old, unusable junk that doesnt even compile.
requests for the sourcecode in iron-forum are consequently silently ignored - why?

and finally, i hate people that take open source software, re-brand id, compile it, distribute it in binary form only and make money by putting some ads on the homepage.
this is just ridiulous ans should definitely not be supportet.

<Iron> because a fork will bring a lot of publicity to my person and
my homepage
<Iron> that means: a lot of money too
<Kmos> rotflol
<Iron> what means rotful?
<mgreenblatt> Iron.. you're a large corporation that can dedicate the
time to support a fork of something as complicated as
chromium?
<Kmos> Iron: google about it
<Iron> yes there is enough time to support it
<jamessan> heh, you're expecting to make lots of money from making a
fork of chromium? that's quite amusing
<Iron> i dont take money for my fork
<Iron> but i have adsense on my page
<Iron> a lot of visitor -> a lot of clicka > a lot of money
<Iron> nobody here trusts google
<Iron> the german people say: google is very evil
<jamessan> yet you use google's adsense

also from http://neugierig.org/software/chromium/ … /iron.html (this is the site of an real chrome developer, not a random user)

now, enough with iron here....
this thread is about the development of an adblocker for chromium and not about discussing iron.

Cheers,
ruth

roy_hu · 2010-05-25 02:02:59

Awesome work. I uploaded your code to http://github.com/wh5a/ChradBlock

Last edited by roy_hu (2010-05-25 03:25:34)

ruth · 2010-05-25 03:50:53

@roy_hu,
hey, nice that you want to help ;-)

anyways, commit access to my own code would be even better ;-)

2nd:
i really wouldnt commit it that way, because tracking the gypi file and the *Loader.cpp files is... suboptimal.
my recommendation:
create a patch for *Loader (only 2 includes and 2 hooks (shouldbeBlocked(url)))
keep the AdBlock.cpp and Adblock.h as regular files in the repo.
so a better structure would probably be:
rootfolder
*AdBlock.cpp
*AdBlock.h
*adblock_hooks.patch (against DocLoader.cpp and FrameLoader.cpp)

i think, that would be better, because Docloader.cpp and FrameLoader.cpp can come still from the main chromium repo and doesnt need to be synced manually (the patch can be maintained very easy)

what do you think?
did that already:
http://github.com/deinococcus/ChradBlock
agreed?

edit:
oh, and a build process (e.g. PKGBUILD, EBUILD, ...) can just checkout the repo, copy the 2 files into the original tree and finally apply the hooks_patch. done.
i really dont want to keep Docloader.cpp and FrameLoader.cpp in sync with upstream... way too much work and will probably fail frequently
(a patch would apply with a fuzz, an out of sync file would break the build process - not so good..)

Last edited by ruth (2010-05-25 03:54:40)

drcouzelis · 2010-05-25 05:07:13

Thanks, I learned a lot by reading your links.

I think the only reason I stopped using Chromium was because I didn't want to download ads. I will follow this thread.

roy_hu · 2010-05-26 03:24:18

ruth wrote:

@roy_hu,
my recommendation:
create a patch for *Loader (only 2 includes and 2 hooks (shouldbeBlocked(url)))
keep the AdBlock.cpp and Adblock.h as regular files in the repo.
so a better structure would probably be:
rootfolder
*AdBlock.cpp
*AdBlock.h
*adblock_hooks.patch (against DocLoader.cpp and FrameLoader.cpp)
i think, that would be better, because Docloader.cpp and FrameLoader.cpp can come still from the main chromium repo and doesnt need to be synced manually (the patch can be maintained very easy)
what do you think?
did that already:
http://github.com/deinococcus/ChradBlock
agreed?

Yes, I agree that your way of structuring code is better. In fact, I thought of doing exactly the same thing. But by managing the files in place, I can do the development faster. I'm going to fork your github project for easier collaboration, but I'll probably continue managing the files however I like it:)

roy_hu · 2010-05-26 03:28:34

Wow, I just noticed that I was granted write access. Thanks!

roy_hu · 2010-05-26 05:34:39

@ruth, too bad it only works with --no-sandbox. Any plans to attack this? How hard is it to implement the proper IPC so that the ad-list is loaded in the browser process, which then communicate with the renderer via IPC?

EDIT: I'm working on the IPC stuff. I think I'm on the right track.

Last edited by roy_hu (2010-05-26 08:21:50)

roy_hu · 2010-05-26 09:05:22

I created a branch on github called "ipc". It's incomplete yet.

ruth · 2010-05-26 13:19:37

hi roy_hu,
very, very, good work on this ;-)

well, the reason i released this stuff at this stage, to tell you the truth, is that i have only very limited time to continue on this stuff... ;-)
on the other side, i thought it is relevant enough for other people to help me, you have done that, thank you a lot ;-)

funny enough, reading you README, you commited
i compared it with my own update script:
update.sh

#!/bin/sh
export PATH=`pwd`/depot_tools:"$PATH"

# remove files from the tree,
# triggers a re-fetch so we are definitely up to date
rm -rfv `pwd`/src/third_party/WebKit/WebCore/loader/AdBlock.cpp
rm -rfv `pwd`/src/third_party/WebKit/WebCore/loader/AdBlock.h
rm -rfv `pwd`/src/third_party/WebKit/WebCore/loader/FrameLoader.cpp
rm -rfv `pwd`/src/third_party/WebKit/WebCore/loader/DocLoader.cpp
rm -rfv `pwd`/src/third_party/WebKit/WebCore/WebCore.gypi

gclient sync --force --nohooks
# gclient sync --force --nohooks --revision src@47915

# make a of the original files
# in case the patch needs regeneration
cp -v `pwd`/src/third_party/WebKit/WebCore/loader/FrameLoader.cpp FrameLoader.cpp
cp -v `pwd`/src/third_party/WebKit/WebCore/loader/DocLoader.cpp DocLoader.cpp
cp -v `pwd`/src/third_party/WebKit/WebCore/WebCore.gypi WebCore.gypi

# link Blocker to the tree
ln -s `pwd`/ChradBlock/AdBlock.cpp `pwd`/src/third_party/WebKit/WebCore/loader/
ln -s `pwd`/ChradBlock/AdBlock.h `pwd`/src/third_party/WebKit/WebCore/loader/

# apply patch
cd src && patch -p1 -i "../ChradBlock/ChradBlock_hooks.patch"

but, i am still on board and will definitely continue... ;-)

Cheers,
ruth

Last edited by ruth (2010-05-26 13:41:42)

ruth · 2010-05-26 15:16:26

added a README that describes how to setup this thing ;-)

this should definitely help people to get started...

Cheers,
ruth

Last edited by ruth (2010-05-26 15:16:42)

praavDa · 2010-05-26 18:43:16

Thank You for sharing, I will definitely try this at home.

As for me, for blocking ads I am using:

- http://www.mvps.org/winhelp2002/hosts.htm
- adblock
- flashblock.

I think that adblock is great, but modified host file is a charm. It is regularly updated and does the same what Your patch - stops ads from loading by mapping lots of popular phishing / ad sites to 127.0.0.1.

Last edited by praavDa (2010-05-26 18:45:05)

ruth · 2010-05-26 19:09:01

@praavDaq
and how would you block access to
http://graphics8.nytimes.com/adx/images … banner.jpg
or
http://graphics8.nytimes.com/ads/house/NYT_logo_red.jpg
with host based blocking?
of course, you could write
127.0.0.1 graphics8.nytimes.com
now, what should i do if i want to watch that video:
http://graphics8.nytimes.com/bcvideo/1. … e=homepage
see?
2nd:
you have to blacklist _every_single_ hostname in the /etc/hosts file
with regex based adblocking you just block all requests to urls, that have */ads/* in it for example or */banners/* or something; much much better...
no, regex based adblocking is far more superior, there's a reason why ablock plus exists, trust me ;-)

if host based blocking is enough for you - fine ;-)
it is _not_ enough for many other people...

Last edited by ruth (2010-05-26 19:13:47)

roy_hu · 2010-05-27 02:43:01

Blocking is purely url-based, so element hiding isn't supported, is it?

roy_hu · 2010-05-27 02:43:42

After spending more time on your code, I made a few comments on your regular expressions at github. Please check them out.

roy_hu · 2010-05-27 04:27:46

I've finished the IPC-based implementation. It's available at http://github.com/deinococcus/ChradBlock/tree/ipc . You may want to comment out some output statements. If you can build this stuff you should be capable to do it. Cheers!

Last edited by roy_hu (2010-05-27 04:28:51)

roy_hu · 2010-05-27 04:28:18

@ruth, Please scrutinize my code and let me know if you want to switch to my codebase...

praavDa · 2010-05-27 04:32:41

@ruth - You are totally right and I agree with You - I just said, that host based blocking is a great tool for big share of ad providing sites. The rest is up to {ad,flash}block + the tool You had provided.

I hate ads and as I said before - thanks for sharing.

ruth · 2010-05-27 14:27:23

@roy_hu:
no, no element hiding yet - you know that already... ;-)
merged the ipc branch, works like a charm...

Cheers,
ruth

roy_hu · 2010-06-17 14:39:35

A native, extensible solution is now being worked on: http://code.google.com/p/chromium/issue … =35897#c63

Arch Linux

#1 2010-05-24 20:13:18

true Adblocking with chromium

#2 2010-05-24 21:19:54

Re: true Adblocking with chromium

#3 2010-05-24 21:35:06

Re: true Adblocking with chromium

#4 2010-05-24 21:49:11

Re: true Adblocking with chromium

#5 2010-05-24 22:41:21

Re: true Adblocking with chromium

#6 2010-05-25 02:02:59

Re: true Adblocking with chromium

#7 2010-05-25 03:50:53

Re: true Adblocking with chromium

#8 2010-05-25 05:07:13

Re: true Adblocking with chromium

#9 2010-05-26 03:24:18

Re: true Adblocking with chromium

#10 2010-05-26 03:28:34

Re: true Adblocking with chromium

#11 2010-05-26 05:34:39

Re: true Adblocking with chromium

#12 2010-05-26 09:05:22

Re: true Adblocking with chromium

#13 2010-05-26 13:19:37

Re: true Adblocking with chromium

#14 2010-05-26 15:16:26

Re: true Adblocking with chromium

#15 2010-05-26 18:43:16

Re: true Adblocking with chromium

#16 2010-05-26 19:09:01

Re: true Adblocking with chromium

#17 2010-05-27 02:43:01

Re: true Adblocking with chromium

#18 2010-05-27 02:43:42

Re: true Adblocking with chromium

#19 2010-05-27 04:27:46

Re: true Adblocking with chromium

#20 2010-05-27 04:28:18

Re: true Adblocking with chromium

#21 2010-05-27 04:32:41

Re: true Adblocking with chromium

#22 2010-05-27 14:27:23

Re: true Adblocking with chromium

#23 2010-06-17 14:39:35

Re: true Adblocking with chromium

Board footer