Blocking Ads using Squid Proxy Server
Squid Proxy Server is one of the famous open source proxy server available and widely used by a lot of enterprise. Squid proxy server not only will help to speed up your corporate network to the Internet by cache the content in the server it self, it also have a lot of useful feature such as filtering the content and the website by using the access control list.
Assuming that you already installing the Squid Proxy server and it’s up and running in your environment. This post will focus on how to get the list of ad network domain name and block it at your proxy server so that it will not bordering your user in the network.
Basically there are 3 step to archive this blocking of the ad network as well as auto update the ads network domain list.
- Add the access control list to your squid config
- Create the advertisement network domain list
- Grab the advertisement network domain list from the free
Add the ACL to squid config
Normally the config file for the squid server will be located in /etc/squid/squid.conf, add the following code anywhere in the config file.
The first line below is a comment and reminder where you are getting your list from. The second line is the regular expression that reads the “/etc/squid/ad_block.txt” file when the squid daemon loads or when you reconfigure the daemon with “squid -k reconfigure”. The next line instructs squid to deny access to those ips in the list from clients using the squid proxy. The last line (deny_info) is optional, it just sends back a tcp rest to the client instead of sending an informational error page. You may want this option if you do not want clients provided with any info about your proxy or why the error was triggered.
## disable ads ( http://pgl.yoyo.org/adservers/ ) acl ads dstdom_regex "/etc/squid/ad_block.txt"
http_access deny ads #deny_info TCP_RESET ads
Downloading the ads network domain list
The next step is to fetch the list of known advertising hostnames and save them to a file so squid can read it. The following script uses curl to download the list from pgl.yoyo.org and save the list to a file in /etc/squid/ad_block.txt. The last line in the script tells squid to re-read the ad_block.txt list after the file is downloaded to load in any new ad servers.
Create the file ad_servers_newlist.sh in /etc/squid/ directory.
$ sudo touch ad_servers_newlist.sh # create the .sh file $ sudo chmod +x ad_servers_newlist.sh # add the execute permission#### Calomel.org ad_servers_newlist.sh #
## get new ad server list $ curl -sS -L --compressed "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml&showintro=0&mimetype=plaintext" > /etc/squid/ad_block.txt ## refresh squid
$ /usr/local/sbin/squid -k reconfigure
Schedule the download
Last step is to run the schedule using cron job to update the list every few days (as per you like). This site ( pgl.yoyo.org) normally will update their list around 3 days on average. By scheduling the download to be trigger every 3 days will make sure you are getting the latest list to block the ads network.
The following cron job is running every 3 days at 12:00am (00:00).
$ crontab -e # open the crontab configuration file#minute (0-59)
#| hour (0-23)
#| | day of the month (1-31)
#| | | month of the year (1-12 or Jan-Dec)
#| | | | day of the week (0-6 with 0=Sun or Sun-Sat)
#| | | | | commands
#| | | | | |
#### refresh squid's anti-ad server list
0 0 * * */3 /etc/squid/ad_servers_newlist.sh >> /dev/null 2>&1
By filtering the ads network at the proxy end, you are not only save up the Internet bandwidth, but also reduce the possible that the user in your network speeding time to load the ads or clicking around the attractive ads to browse other site.
Originally published at https://tech.david-cheong.com on May 12, 2020.