Trying to set up a transparent proxy

aquavitae · 2010-05-03 07:28:38

At work, we connect to the internet through a proxy which requires a username and password. To avoid having to set this up (and maintain it) for every app that connects to the internet (e.g. pacman, kde, gnome, firefox, etc) I've configured squid as a transparent proxy that simply forwards requests to the network proxy with the appropriate credentials. The trouble is that it only works for http - I can't get it to work for any other port.

Please could someone give me some advice on how to do this. My config files are below:

squid.conf

acl all src all
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80        # http
acl Safe_ports port 21        # ftp
acl Safe_ports port 443        # https
acl Safe_ports port 70        # gopher
acl Safe_ports port 210        # wais
acl Safe_ports port 1025-65535    # unregistered ports
acl Safe_ports port 280        # http-mgmt
acl Safe_ports port 488        # gss-http
acl Safe_ports port 591        # filemaker
acl Safe_ports port 777        # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow !Safe_ports
http_access allow CONNECT !SSL_ports
http_access allow all
icp_access allow all
http_port 127.0.0.1:3128 transparent
cache_peer 192.168.10.50 parent 8080 8080 login=username:password default no-query
never_direct allow all
hierarchy_stoplist cgi-bin ?
access_log /var/log/squid/access.log squid
refresh_pattern ^ftp:        1440    20%    10080
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern -i (/cgi-bin/|\?) 0    0%    0
refresh_pattern .        0    20%    4320
acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
upgrade_http0.9 deny shoutcast
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
coredump_dir /var/cache/squid
shutdown_lifetime 1 seconds

iptables/squid.rules

iptables -t nat -A OUTPUT -p tcp --dport 80 -m owner --uid-owner proxy -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport 3128 -m owner --uid-owner proxy -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport 80 -j REDIRECT --to-ports 3128

Last edited by aquavitae (2010-05-03 07:44:46)

Sin.citadel · 2010-05-03 08:36:55

specific network based firewalls like MS ISA provide a special client which automatically attaches credentials for the user for every tcp connection, there is no automatic way to do this, your network admin can probably help you more as a special client is required.

aquavitae · 2010-05-03 09:42:41

I need MS ISA for ftp connections, but not https. I think can see why it won't work - iptables doesn't know forward https to the proxy. But I don't know how to fix it.

Allan · 2010-05-04 04:15:37

aquavitae wrote:

iptables -t nat -A OUTPUT -p tcp --dport 80 -m owner --uid-owner proxy -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport 3128 -m owner --uid-owner proxy -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport 80 -j REDIRECT --to-ports 3128

I am using the same setup. I use something on another port and just had to add:

iptables -t nat -A OUTPUT -p tcp --dport XXXX -m owner --uid-owner proxy -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport XXXX -j REDIRECT --to-ports 3128

Where XXXX was the port.

Be warned that I have no idea what I am doing...

aquavitae · 2010-05-04 06:50:47

Thanks for the suggestion, but it didn't work.

To test it, I'm trying to connect to https://help.ubuntu.com, and using kopete to connecto to gtalk (port 5223). Neither work. I also tried running "nmap localhost", which gives the following output:

Starting Nmap 5.21 ( http://nmap.org ) at 2010-05-04 08:48 SAST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00047s latency).
rDNS record for 127.0.0.1: localhost.localdomain
Not shown: 995 closed ports
PORT      STATE SERVICE
80/tcp    open  http
631/tcp   open  ipp
3128/tcp  open  squid-http
8010/tcp  open  xmpp
50001/tcp open  unknown

Nmap done: 1 IP address (1 host up) scanned in 0.07 seconds

So it looks like I need to open the other ports somehow, but how?

Sin.citadel · 2010-05-04 07:32:21

you cant transparent proxy https connections, it destroys the inherent security present in https, the only way to connect https is to use your gateway's proxy server in the firefox/IE/browser settings, or simply forward https connections, you cant transparently proxy https connections.

aquavitae · 2010-05-04 07:53:06

That's what I thought, but how do I forward them? I don't really want to have to change browser settings for it. Would I do the same for other ports then too?

Sin.citadel · 2010-05-04 10:36:39

by forward, i mean that your arch machine will simply forward the https connection on port 443 to your gateway, at which time it will be denied because you need authorization that the network client provides,there is no way except to set up the proxy by which you can access https sites, i recommend you use foxyproxy with firefox to switch the settings when your in office/home. also, as to your other tcp ports situation, if your gateway support SOCKS5 proxy, you can transparently proxy your other connections to it (SOCKS5 supports authentication) with a program like tsocks.

aquavitae · 2010-05-05 12:56:57

Ok, Thanks for the info.

Another idea then. If its just the transparent proxy that's causing the problem, how about removing the transparency. Presumably I could set my system up to access the internet via localhost:3128. Then I would need to set up two squid.conf files, one for home and one for work, and write a simple script to switch between the two. Would this work? And could I still use it transparently for apps that only send http requests (e.g. pacman)?

Last edited by aquavitae (2010-05-05 12:58:24)

Sin.citadel · 2010-05-05 13:27:56

when you are using squid transparently, it doesnt mean that squid stops listening at port 3128, it means that if a request comes only to port 80 (or any that you specify) it will transfer it to squid on port 3128, if you specify a http proxy in your application with address localhost and port 3128, it will work as well. so you dont need to remove the transparent option.

as for two config files, yes it will work, but in your script rather than restarting squid after a config change, use squid -k reconfigure instead.

aquavitae · 2010-05-06 05:30:23

Thanks for the help! I've done that and it seems to be working (mostly) so far. Just need to get home to try it there too. I'm still having problems with gtalk though - I've tried specifying ports 5223, 5222 and 443 in kopete, but it won't connect. Can you give my any hints...? I'm not even sure how to find out where the problem is. I know that the network proxy doesn't block it because I can use the google client in Windows,
[EDIT: Just done a bit of googling and it seems thing is a bug in kopete - it doesn't work with a proxy. So can I use iptables to forward port 5223 through squid?]

The next step in my network config is to try to get my squid configuration script to run automatically depending which network I'm connected to. In fact, there are a few other things I'd like to do automatically when I connect (e.g. mount samba shares). Any suggestions on how I would do this?

Last edited by aquavitae (2010-05-06 05:42:04)

Allan · 2010-05-06 05:40:10

Apparently, netcfg should be able to help you run things based on which network you connect to.

aquavitae · 2010-05-06 05:45:04

Yes, I know about netcfg, but my connections are all working quite well with networkmanager, and I want it to run automatically. From what I understand, its still necessary to run "netcfg <profile>" to connect to a network. Since the connection it already happening, I could just as easily run my own script instead. Maybe I'm wrong about this?

Sin.citadel · 2010-05-06 06:37:12

if your network uses dhcp, and you have different ip subnets for each network, you can create a script that parses the ifconfig <interface> output and runs your script based on the ip address assigned, you can then add it to your rc.local file for automation.

aquavitae · 2010-05-06 08:37:18

Thanks. I did that (although slightly differently), but I put it in /etc/NetworkManager/dispatcher.d. It turns out that networkmanager will run any scripts in this dir whenever a network event happens. So my final solution:

/etc/squid/squid.work.conf as in my first post
/etc/squid/squid.home.conf exactly the same with without the line contiaining the credentials
iptables configured as in my first post

/etc/NetworkManager/dispatcher.d/load_network:

#!/bin/sh

# First argument is the network interface
eth=$1
# Second argument is the status
stat=$2

# Run whenever a network interface is up
if [[ $stat == "up" ]]
then

  # Get location by checking ip address (set by dhcp)
  if [[ -n /usr/sbin/ip add show $eth to 10.10.103.0/23 ]]
  then
    loc=work
  else
    loc=home
  fi

  # Create squid.conf symlink to appropriate file
  rm -f /etc/squid/squid.conf
  ln -s /etc/squid/squid.$loc.conf /etc/squid/squid.conf
  # Reconfigure squid
  squid -k reconfigure

  # Mount all samba shares (specified in fstab)
  mount -a

fi

Last edited by aquavitae (2010-05-07 09:37:14)

Arch Linux

#1 2010-05-03 07:28:38

Trying to set up a transparent proxy

#2 2010-05-03 08:36:55

Re: Trying to set up a transparent proxy

#3 2010-05-03 09:42:41

Re: Trying to set up a transparent proxy

#4 2010-05-04 04:15:37

Re: Trying to set up a transparent proxy

#5 2010-05-04 06:50:47

Re: Trying to set up a transparent proxy

#6 2010-05-04 07:32:21

Re: Trying to set up a transparent proxy

#7 2010-05-04 07:53:06

Re: Trying to set up a transparent proxy

#8 2010-05-04 10:36:39

Re: Trying to set up a transparent proxy

#9 2010-05-05 12:56:57

Re: Trying to set up a transparent proxy

#10 2010-05-05 13:27:56

Re: Trying to set up a transparent proxy

#11 2010-05-06 05:30:23

Re: Trying to set up a transparent proxy

#12 2010-05-06 05:40:10

Re: Trying to set up a transparent proxy

#13 2010-05-06 05:45:04

Re: Trying to set up a transparent proxy

#14 2010-05-06 06:37:12

Re: Trying to set up a transparent proxy

#15 2010-05-06 08:37:18

Re: Trying to set up a transparent proxy

Board footer