Caching via Squid Proxy on pfSense

Inxsible · 2017-10-04 16:12:15

hello,

I am setting up a Squid Transparent proxy on my pfSense box and would like to cache the common packages across my different Archlinux machines (I have 4). Doing this would allow me to download the common packages only once. My question is, do all these machines have to be using the same mirror in order to fetch the package from the cache instead of fetching it again from the interwebs?

Thanks,

qinohe · 2017-10-04 23:15:44

hallo
Yes probbably, since they are using some sort of url + hash system to check, see https://wiki.squid-cache.org/SquidFaq/CacheDigests
Would it not be easier to set up a local repo?
Me I,m running pfsense myself and threw squid off as I don't see the bennefit + most of the internet today is https and who wants a man in the middle even if it's your own network, just my 2 cents.

Welcome,

Inxsible · 2017-10-09 17:01:29

qinohe wrote:

hallo
Yes probbably, since they are using some sort of url + hash system to check, see https://wiki.squid-cache.org/SquidFaq/CacheDigests
Would it not be easier to set up a local repo?
Me I,m running pfsense myself and threw squid off as I don't see the bennefit + most of the internet today is https and who wants a man in the middle even if it's your own network, just my 2 cents.
Welcome,

The primary reason I was setting up Squid was to use SquidGuard. I intend to use SquidGuard to block adult websites and also block ads. It is my understanding that SquidGuard requires a working Squid setup. Since I would already have Squid installed, I thought I might be able to use it as a transparent proxy to cache packages etc.

Is this not a good idea?

Last edited by Inxsible (2017-10-09 17:02:47)

qinohe · 2017-10-10 11:43:58

Well I think squid is a nice proxy and tool to check http traffic but as soon as it comes to https you would have a man in the middle situation. This is because you're decrypting the stream before it reaches it's endpoint. I personnaly gave up on squid because the whole internet is https(almost) + my connection is not capped, is yours?. You can do what I did, use 'hosts' lists on every machine in my network + snort on pfsense to track the real bad traffic. This doesn't prevent users who don't have such hosts list to go to certain adresses, unless you would put that list on pfsense. Pfsense has pfblocker too,though, I have minimal knowledge of this tool. Hope this helps somehow.

edit:hosts lists are not bound to http, see https://en.wikipedia.org/wiki/Hosts_(file)
Atool to create a hosts list is hosts-update which is in AUR. There are numerous lists out there on the net with some overlap or specific, for a windows machine eg.
There is probably a penalty for using a very big hosts list if your machine is slow, but I haven't noticed even on a Pi. My demands are low...

Last edited by qinohe (2017-10-10 12:13:46)

Inxsible · 2017-10-13 15:52:30

Fair enough. After trying with Squid & Squidguard I have now moved on to pfBlockerNG. I realized that the caching was not giving me much and I didn't want to run Squid just to be able to run Squidguard. Also it still wouldn't block many domains. Either that was due to my mis-configuration or something else.

I haven't set up pfBlockerNG fully yet, but I have noticed a trend in pfSense and it's packages. Things seem to be unnecessary complicated -- especially for a noob in networking like me. There are certain terms that they use in the options that I have to sit and google before understanding what it means and what will happen if I enable or disable them. The pfSense forums aren't much help either because they aren't as active as what I am used to and also if you ask very basic questions (as per the forum users), you don't really get any answers.

It was the same with Squid and Squidguard. I had to follow some online tutorials in setting certain things up, but wasn't understanding why they are doing what they are doing. I am not comfortable with that aspect. Maybe over time, I will get some more experience with pfSense and the packages that I want to use.

My main aim is to use pfSense as my router, use it as a VPN client for my VPN provider and have it block adverts and certain domains. I have achieved the first 2. I just need to be able to block the adverts and porn/gambling/violence domains

Last edited by Inxsible (2017-10-13 15:53:22)

qinohe · 2017-10-15 13:03:00

Your observasitions about pfense are pretty correct I think, it can be cumbersome to find the info you need if you have little knowledge about the subject, I found that about openvpn on PF, that will get better in time if you stay with PF . First pfsense is not free, it has a income model and one of them is the gold subscription which provides you with the 'pfsense guide/ebook', with pfblockerng in it?? (I'm not a subscriber). Second, you need to know the basics of routing and firewalls and the jargon used is making that clear to you. You know it is a thin line between a fence and a hole!
Btw. there is a multi thread about pfblocker but I think you found that alraedy. https://forum.pfsense.org/index.php?board=70.0

I have used squid+guard for years on pfsense until a few years back the https was trending. After this I started using hosts list again, which I did before I used squid. Mostly I create 1 list and distribute that around the network.
My pfsense box is not that differnet from yours, I use it for snort, ups host, openvpn routing/firewall.

Just try pfblockerng for a while it should do what you want, it's url/IP based not protocol/port dependent. I'm curious about your findings. I'll stick with hosts lists for now, I almost never see adverts/violence btw. The pron is not actively tracked on this network, there is no need for that ATM.

Arch Linux

#1 2017-10-04 16:12:15

Caching via Squid Proxy on pfSense

#2 2017-10-04 23:15:44

Re: Caching via Squid Proxy on pfSense

#3 2017-10-09 17:01:29

Re: Caching via Squid Proxy on pfSense

#4 2017-10-10 11:43:58

Re: Caching via Squid Proxy on pfSense

#5 2017-10-13 15:52:30

Re: Caching via Squid Proxy on pfSense

#6 2017-10-15 13:03:00

Re: Caching via Squid Proxy on pfSense

Board footer