SetupSquidTransparentProxy

2009-12-30T00:24:28Z

AndrewColes: Added details on how to do ad-blocking on the pre using the squid proxy

= Squid Transparent Proxy on Palm Pre =

Ok, someone asked for it, so I figured it out, how to do it for fun and profit:

== Installing Squid ==

First, you need to have root access to your device. How to get root, is described elsewhere in this wiki. It would be a good idea to know a little about linux and networking too. '''Be warned: this can make your networking unfunctional!'''

Make your root disk writable:

<source lang="text">
cd / && mount -v -o remount,rw /
</source>

Login to the pre console and type this command to install the squid optware package

<source lang="text">
root@palm-webos-device: # ipkg-opt install squid
Installing squid (2.6.21-2) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/cs08q1armel/cross/unstable/squid_2.6.21-2_arm.ipk
Configuring squid
create default cache and logs dir
chown: unknown user/group nobody:nobody
chown: unknown user/group nobody:nobody
FATAL: Could not determine fully qualified hostname.Please set 'visible_hostname'

Squid Cache (Version 2.6.STABLE21): Terminated abnormally.
CPU Usage: 0.020 seconds = 0.010 user + 0.010 sys
Maximum Resident Size: 0 KB
Page faults with physical i/o: 0
Segmentation fault

You should review the configuration file /opt/etc/squid/squid.conf,
make any necessary change, and complete the install by running -
/opt/etc/init.d/S80squid start

Successfully terminated.
</source>

Ignore the errors for the moment, we'll get to it soon.

== Create a basic proxy config ==

Change to the directory '''/opt/etc/squid''' and move the file '''squid.conf''' which is already there to somewhere else. Now create a new squid.conf, which should have the following contents:

<source lang="text">
visible_hostname localhost
http_port 3128 transparent
hierarchy_stoplist cgi-bin ?
access_log /var/cache/squid/squid.log squid
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
coredump_dir /var/cache/squid
cache_dir ufs /var/cache/squid 20 5 5
access_log /var/cache/squid/squid.log
cache_log /var/cache/squid/cache.log
cache_store_log /dev/null

acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

acl enemy dstdomain www.apple.com
http_access deny enemy

http_access allow all
icp_access allow all
</source>

Basically this is a config for a transparent proxy, thus a browser will be redirected to it without knowing that it is actually connected to a proxy. We must configure it this way because the WebOS browser cannot be configured to use a proxy.

Note that we have added a single ACL here, which denies access to some random site *g*. Any other requests will be allowed. SSL (HTTPS) will not be passed through the proxy because this requires the browser to connect to it using a proxy request.

== Prepare the directories for squid ==

Squid needs a directory where to store it's files, as content it caches and logfiles. You may consider to disable caching and logging. But caching may be a good idea if your traffic is limited, so you'll save some. Logging is always usefull for debugging. If you want to turn it off, use '''/dev/null''' as target for the logfiles.

<source lang="text">
mkdir /var/cache/squid
chown nobody /var/cache/squid
</source>

Next you need let squid to prepare the directory structure using the -z option:

<source lang="text">
root@palm-webos-device: # squid -z -f squid.conf
2009/11/26 23:19:29| Creating Swap Directories
</source>

You should not see any errors here!

== Starting squid ==

The installer already added it to the startup daemons, now we will start it manually:

<source lang="text">
/opt/etc/init.d/S80squid start
</source>

Look in '''/var/cache/squid/cache.log''' for any errors. Use ps to see if it really runs. You may consider to install '''lynx''' and test the proxy:

<source lang="text">
ipkg-opt install lynx
export http_proxy=http://localhost:3128
lynx www.google.com
</source>

If you see a google textmode site, it works.

== Redirect Web Traffic to the Proxy ==

Now comes the ugly part. I'll not explain it in detail: we install some iptable Rules. We tell iptables to forward any traffic destined to tcp port 80 to localhost port 3128 (this is where squid listens for incoming connections). To avoid that the requests of the proxy itself are forwarded to localhost (which would create a loop), we add a rule telling iptables to not forward web traffic if it were generated by user nobody (the user squid runs as):

<source lang="text">
iptables -t nat -A OUTPUT -p tcp --dport 80 -m owner --gid-owner 65534 -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport 80 -j DNAT --to-destination 127.0.0.1:3128
</source>

You should not get any response from iptables. Let's see if the rules were added correctly:
<source lang="text">
root@palm-webos-device: # iptables -nL -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination

Chain POSTROUTING (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 owner GID match65534
DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 to:127.0.0.1:3128
</source>

You may add the iptables commands to the squid startup script (/opt/etc/init.d/S80squid).

== See it working ==

So, let's see how it works. Fire up the browser and enter the url of your favourite enemy vendor (the one we configured above):

[[Image:Squid-access-denied-palm-pre.png]]

== Blocking Adverts ==

Now squid is set up, we can use it for ad blocking. There are two steps to this:

=== Obtain a block list ===

An excellent ad block list is maintained at [http://pgl.yoyo.org/as/]. We need to download it in a suitable format for squid, which can be done by running the following command:

<source lang="text">
wget -q -O - 'http://pgl.yoyo.org/as/serverlist.php?hostformat=squid-dstdom-regex;showintro=0' | grep -v ">" | grep "." > /opt/etc/squid/blocklist
</source>

(Technical details: the wget command downloads the URL given in single quotes, then we strip the HTML mark-up and blank lines using grep. The filename at the end is where the black list will be stored.)

=== Configure squid to use the block list ===

Now we need to tell squid to block the list of URLs in the file we've just created. Open the ''/opt/etc/squid/squid.conf'' file we created above, and scroll down to where it reads:

<source lang="text">
acl enemy dstdomain www.apple.com
</source>

Replace this line with:

<source lang="text">
acl enemy dstdom_regex "/opt/etc/squid/blocklist"
</source>

Finally, restart squid:

<source lang="text">
/opt/etc/init.d/S80squid restart
</source>

To test it's working, open a browser and try to visit ''ads.msn.com'' (one of the blocked URLs). The error given should be similar to the screenshot given above.

If in the future you need to update the ad blocking list, run the commands:

<source lang="text">
cd / && mount -v -o remount,rw /
wget -q -O - 'http://pgl.yoyo.org/as/serverlist.php?hostformat=squid-dstdom-regex;showintro=0' | grep -v ">" | grep "." > /opt/etc/squid/blocklist
/opt/etc/init.d/S80squid restart
</source>

For further details of filtering URLs using squid, please refer to the [http://www.squid-cache.org/Doc/ excelent documentation of the squid cache project].

WebOS Internals - User contributions [en]

SetupSquidTransparentProxy