You are not logged in.

#1 2009-09-15 09:52:59

kriz
Member
Registered: 2009-06-29
Posts: 96

lil' script to update adblockfilter, adblocking via /etc/hosts file

hi, i've recently changed to adblocking via the hosts file (which works great btw), but i was missing  filtersetupdating like in firefox, so i've created with my limited scripting skills this one...

# lil' script to update /etc/hosts adblock-filter
#
#hosts adblock filter taken from this site...
wget --directory-prefix=/tmp http://www.mvps.org/winhelp2002/hosts.txt
#Backup /etc/hosts to /tmp
cp /etc/hosts /tmp
#standard static hosts file
echo '# /etc/hosts: static lookup table for host names' > /etc/hosts
echo '#' >> /etc/hosts
echo '#<ip-address>    <hostname.domain.org>    <hostname>' >> /etc/hosts
echo '127.0.0.1        localhost.localdomain    localhost' >> /etc/hosts
#add custom statc host configuration here 
echo ' ' >> /etc/hosts
echo '###Ad-Blocking###' >> /etc/hosts
cat /tmp/hosts.txt >> /etc/hosts
echo '# End of file' >> /etc/hosts
rm /tmp/hosts.txt

enjoy!


„Je verdinglichter die Welt, je dichter das Netz, das der Natur überworfen wurde, desto mehr beansprucht ideologisch das Denken, das jenes Netz spinnt, seinerseits Natur, Urerfahrung zu sein." Theodor W. Adorno [aus: Wozu noch Philosopie]

Offline

#2 2009-12-25 18:44:13

scragar
Member
Registered: 2009-07-14
Posts: 108

Re: lil' script to update adblockfilter, adblocking via /etc/hosts file

Sorry to bump this thread, but I was looking for a solution to something else.

I made a few modifications to this code, added some basic error checking, and made it more efficient, hope you enjoy.

#!/bin/bash

ADBLOCK='http://www.mvps.org/winhelp2002/hosts.txt';
HOSTSHEAD=/etc/hosts_head;

if [ $(whoami) != "root" ]; then
  echo "Please run as root." >&2
  exit 1
fi
if [ ! -f "${HOSTSHEAD}" ]; then
  echo "Header file for hosts missing." >&2
  exit 2
fi

if [ -z "$(curl -I "${ADBLOCK}" 2>&1 | grep '200 OK' )" ]; then
  echo "Error, return code for Adblock lists is not 200." >&2
  echo "This typically indicates an error, rather than anything else." >&2
  exit 3
fi

cp "${HOSTSHEAD}" /etc/hosts
wget -O - "${ADBLOCK}" | sed /localhost/d >> /etc/hosts

EDITED to clean up a little.

Last edited by scragar (2009-12-25 21:26:21)

Offline

#3 2009-12-25 20:37:10

Meyithi
Member
From: Wirral, UK
Registered: 2009-06-21
Posts: 550
Website

Re: lil' script to update adblockfilter, adblocking via /etc/hosts file

Cool, have been looking into this myself but wasn't happy with the duplicate

127.0.0.1 localhost

that is always present in the downloaded file.

I've put this in at the end to remove the duplicate "127.0.0.1 localhost"

sed -i -e 's-^127.0.0.1  localhost-#-' /etc/hosts

Seems to work, is probably a more elegant way though!


The mind roams more freely in empty rooms.
dwm - colours - ncmpcpp - system
irc://irc.freenode.net:meyithi

Offline

#4 2009-12-25 20:54:03

scragar
Member
Registered: 2009-07-14
Posts: 108

Re: lil' script to update adblockfilter, adblocking via /etc/hosts file

Meyithi wrote:

Cool, have been looking into this myself but wasn't happy with the duplicate

127.0.0.1 localhost

that is always present in the downloaded file.

I've put this in at the end to remove the duplicate "127.0.0.1 localhost"

sed -i -e 's-^127.0.0.1  localhost-#-' /etc/hosts

Seems to work, is probably a more elegant way though!

Just edit the last line of my edit:

wget -O - "${ADBLOCK}" | sed /localhost/d >> /etc/hosts

Offline

#5 2009-12-25 21:07:26

Meyithi
Member
From: Wirral, UK
Registered: 2009-06-21
Posts: 550
Website

Re: lil' script to update adblockfilter, adblocking via /etc/hosts file

Cheers!


The mind roams more freely in empty rooms.
dwm - colours - ncmpcpp - system
irc://irc.freenode.net:meyithi

Offline

#6 2010-03-20 03:29:44

supulton
Member
Registered: 2008-12-31
Posts: 58

Re: lil' script to update adblockfilter, adblocking via /etc/hosts file

Great script. smile

Offline

#7 2012-02-15 09:11:12

ontobelli
Member
From: Mexico City
Registered: 2011-02-06
Posts: 127

Re: lil' script to update adblockfilter, adblocking via /etc/hosts file

hosts_udate

#!/bin/bash
#
# 2012 Ontobelli for this script

# make hosts temporal directory
HOSTSDIR=~/.hostsupdate
mkdir -p "${HOSTSDIR}"

# make temporary directory
TMPDIR=/tmp/hostsupdate
mkdir -p "${TMPDIR}"

# set output file
OUTPUTFILE="${TMPDIR}/hosts"

# set temporal file
TMPFILE="${TMPDIR}/tmpfile"

if [ ! -f "${HOSTSDIR}/hosts.local" ]; then
 echo "You need to create "${HOSTSDIR}"/hosts.local containing the hosts you wish to keep!"
 exit 0
fi

# download the mvps.org hosts file.
wget -c -O "${HOSTSDIR}/hosts.mvps" "http://winhelp2002.mvps.org/hosts.txt"
# download hpHOSTS
wget -c -O "${HOSTSDIR}/hosts.hphosts" "http://support.it-mate.co.uk/downloads/HOSTS.txt"
# download hpHOSTS Partial
wget -c -O "${HOSTSDIR}/hosts.partial" "http://hosts-file.net/hphosts-partial.asp"
# download hpHOSTS ad/tracking servers
wget -c -O "${HOSTSDIR}/hosts.adservers" "http://hosts-file.net/ad_servers.asp"
# download the pgl.yoyo.org hosts Peter Lowe - AdServers
wget -c -O "${HOSTSDIR}/hosts.yoyo" "http://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext"
# download SysCtl Cameleon hosts
wget -c -O "${HOSTSDIR}/hosts.sysctl" "http://sysctl.org/cameleon/hosts"

# cat entries in a single file
cat "${HOSTSDIR}/hosts.mvps"       > "${TMPFILE}0"
cat "${HOSTSDIR}/hosts.hphosts"   >> "${TMPFILE}0"
cat "${HOSTSDIR}/hosts.partial"   >> "${TMPFILE}0"
cat "${HOSTSDIR}/hosts.adservers" >> "${TMPFILE}0"
cat "${HOSTSDIR}/hosts.yoyo"      >> "${TMPFILE}0"
cat "${HOSTSDIR}/hosts.sysctl"    >> "${TMPFILE}0"

# tabs to space
sed -e 's/	/ /g' "${TMPFILE}0" > "${TMPFILE}1"
# find relevant lines without comments
grep ^127.0.0.1 "${TMPFILE}1" > "${TMPFILE}2"
# remove duplicate spaces
cat "${TMPFILE}2" | tr -s [:space:] > "${TMPFILE}3"
# remove carriage returns
cat "${TMPFILE}3" | tr -d "\r" > "${TMPFILE}4"
# 0.0.0.0 is nicer than constantly knocking on localhosts' door.
sed -e 's/127.0.0.1 /0.0.0.0 /g' "${TMPFILE}4" > "${TMPFILE}5"
# remove inline comments
cut -d ' ' -f -2 "${TMPFILE}5" > "${TMPFILE}6"
# sort blocklist entries and remove duplicates
sort "${TMPFILE}6" | uniq > "${TMPFILE}7"
# remove unneeded blocked sites
grep -Ev ' dl.dropbox.com| host_you_want_to_whitelist' "${TMPFILE}7" > "${TMPFILE}9"

# write the user's hosts.local to head, then the blacklists
cat "${HOSTSDIR}"/hosts.local > "${OUTPUTFILE}"
cat "${TMPFILE}9" >> "${OUTPUTFILE}"
echo -e "# end of file" >> "${OUTPUTFILE}"

# move to /etc/hosts
mv "${OUTPUTFILE}" /etc/hosts

# delete temporary directory
rm -r -f "${TMPDIR}"

hosts.local

# /etc/hosts: static lookup table for host names
#
#<ip>	<hostname.domain.org>	<hostname>
127.0.0.1 localhost.localdomain localhost YOURHOSTSNAMEHERE
::1	        localhost.localdomain localhost YOURHOSTSNAMEHERE
# YOUR PERSONAL list
# blocked list

Create an alias in your ~/.bashrc

alias hu='sudo /root/.hostsupdate/hosts_update'

Run

# hu <enter>

Script and cache must be located in /root/.hostsupdate or modify scrip accordingly

Cheers.

Last edited by ontobelli (2012-02-15 09:15:17)

Offline

#8 2012-02-15 21:11:41

el mariachi
Member
Registered: 2007-11-30
Posts: 595

Re: lil' script to update adblockfilter, adblocking via /etc/hosts file

this is all good, but a simple instance of urxvt takes ages to start with this method. is there a way of telling urxvt to not read the hosts file?

Offline

#9 2012-02-15 21:43:57

vwyodajl
Member
Registered: 2012-01-21
Posts: 183

Re: lil' script to update adblockfilter, adblocking via /etc/hosts file

ontobelli's looks similar to graysky's hosts_update in the AUR.. Very simple and easy to use and converts everything to 0.0.0.0

Offline

#10 2012-02-16 04:01:02

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 19,728

Re: lil' script to update adblockfilter, adblocking via /etc/hosts file

Moderator comment:

Guys:  Watch the age of the thread -- This one has been stagnant for two years. Our Policy.

Closing


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

Board footer

Powered by FluxBB