Squid in Ubuntu Overview

A proxy server is a very useful tool for a network. It is commonly used in computer networks to protect the network from attack, to filter nefarious web content and pages requested by local users, and to speed up the delivery of web pages and web content by caching (storing) commonly requested web pages, documents, and media. Proxy servers are typically implemented on private, local area networks, to filter, protect and cache content requested by users on that network, this is called “proxy” or “transparent proxy.” Proxy servers can also be implemented on the remote side “in-front-of” destination webservers in order to protect those servers by filtering requests, speeding up web page delivery, and caching frequently requested files, this is called “reverse proxy.”

Types of Proxy Servers

Squid is one of the most popular and most used proxy servers in the world. It is free to download, easy to install and it can be implemented on any distribution of Linux. Here are the steps to install and configure Squid on an Ubuntu distribution of Linux.

Steps to install and configure Squid

Open a terminal, and type in the following commands to install Squid

sudo apt-get update
sudo apt-get install squid squid-common

Ways to start and stop Squid

sudo service squid start (stop|restart|status)
sudo /usr/sbin/squid (launch program directly)
sudo pkill -9 squid

Navigate to the Squid folder to find the squid.conf configuration file

cd /etc/squid
ls (you should see the squid.conf file)

Create a backup of the squid.conf file

sudo cp squid.conf squid.conf.bak

For testing purposes open Firefox and set it to send web requests to the Squid Proxy Server (You will need to know your ip address)

ifconfig (write down your inet address e.g. 192.168.1.100)
 

Open Firefox

Edit > Preferences, Advanced > Network Tab > Connection-Settings:

Manual Proxy Configuration:

HTTP Proxy: your IP address or loopback address 127.0.0.1,    Port: 3128
   Click Ok and Close

Now if you try and go to a website like google you should see an ERROR – Access Denied message from Squid (see bottom line). This means that Squid is working by actively denying the traffic.

Now we need to configure Squid to allow web traffic through the proxy server. Open squid.conf in your favorite text editor like gedit, nano, or vi

sudo nano squid.conf

or

sudo gedit squid.conf & (If gedit does not open from the terminal you can open it as root user)
sudo su
gedit squid.conf &

To switch out of root user

su your-username (if you are root user the prompt is a "#" switch back to your user account privilege)

If you chose to open with squid.conf with gedit, then turn on line numbering (Edit > Preferences > View >Display Line Numbers)

Change the name of your Squid Proxy Server, around line 3399, change.

#    TAG: visible_hostname

to

 visible_hostname YourNameProxyServer  

You can configure access rules for your Squid proxy server (lines 331 to 831 are for Access Control). Notice that on lines 606 to 630 the local networks and usable ports (services) are defined. Active configuration lines, are the lines that are not commented out, i.e. they do not start with  a # sign.

To re-enable web access uncomment line 676

#http_access allow localnet

to

http_access allow localnet

To verify the Web is now working, save your changes to the squid.conf file and restart your Squid server.

sudo service squid restart

Now resfresh your Firefox web browser and your homepage should be visible.

Now we can practice writing a custom ACL (access list) in the squid.conf file to block specific domains and websites. We can write our custom ACL at the end of the acl lines around line 631. From an empty line write the following lines to test domain blocking.

acl blocked_websites dstdomain .msn.com .yahoo.com
http_access deny blocked_websites

Now restart your Squid server, and test to see if Squid denies access to your blocked domains/websites in Firefox.

Video Tutorials

In this series of videos, I go through the same process outlined above, to install and configure a Squid proxy server in Ubuntu .

In part 1, I install Squid in Ubuntu, start and stop it, backup the configuration
file, and configure Firefox to use Squid as a proxy server

In part 2, I discuss editing the configuration file

In part 3, I write a custom ACL in the squid.conf file

Install and Configure SquidGuard Overview

It is very useful to be able to block users on your network from accessing millions of websites with nefarious content. A great way to accomplish this is with a proxy server like Squid. Squid is a free and powerful proxy server that is capable of blocking users from accessing web content. A great way of enhancing Squid’s ability to block unwanted websites, domains and IP addresses is to install SquidGuard. SquidGuard is an add-on program for the Squid proxy server (see my previous article on Squid), that’s main purpose is to block unwanted web traffic. SquidGuard works with databases of blacklists to block, filter, and redirect requested URLs and domains. You manually download and add blacklist files to SquidGuard and compile them into the SquidGuard database, then Squid can redirect web requests, checking them against SquidGuard’s database of blacklisted websites, domains and IP addresses. It seems like this process would slow down a network, but SquidGuard is an extremely fast web content filter with the ability to check web requests against millions of blacklisted sites in a matter of seconds. There is great information about SquidGuard’s capabilities on the SquidGuard website, including links to download the program, well written installation and configuration instructions, and links to websites that maintain blacklists.

Steps to manually install SquidGuard in Ubuntu

You can download and install SquidGuard using a package manager program like apt-get or yum, or even a graphical software installer tool like the software center program. Instead, I chose to outline the steps involved in manually downloading and installing SquidGuard.

1. Download the current stable version of SquidGuard at http://www.squidguard.org and save it to your downloads folder.

2. Download the Berkeley DB from Oracle at http://oracle.com. Download version 4.8.30.NC.tar.gz with no encryption and save it to your Downloads folder

3. Open a terminal and navigate to the directory where you downloaded SquidGuard and the BerkeleyDB. You should see the tar.gz files
cd ~/Downloads
ls

4. Decompress the tar.gz files (substitute the file names for the versions you downloaded)
tar -xvzf squidGuard-1.5-beta.tar.gz
tar -xvzf db-4.8.30.NC.tar.gz
ls
You should see two folders one for squidGuard and one for BerkeleyDB (e.g. squidGuard-1.5, and db-4.8.30)

5. Install the Berkeley DB first, since SquidGuard requires it for installation. By default, the Berkeley DB will install itself to a directory in /usr/local/ in a folder named BerkeleyDB.4.8 you will need this information when preparing SquidGuard for installation.
cd db-4.8.30
cd build_unix
../dist/configure
make
sudo make install

6. Install SquidGuard by navigating to the extracted SquidGuard folder and then during the configure process you will pass the configure script the location of the Berkeley DB directory and correctly change the squiduser to ‘proxy’ for Ubuntu. The squiduser and group is typically “squid” in other Linux distributions like Fedora.
cd ~/Downloads/squidGuard-1.5
./configure –with-db=/usr/local/BerkeleyDB.4.8 –with-squiduser=proxy
make
sudo make install

You should get a message that the initial SquidGuard configuration is complete. Congratulation, SquidGuard is successfully installed! Make a note of the directory locations of the SquidGurad db, log, and conf files:
/usr/local/squidGuard/db
/usr/local/squidGuard/log
/usr/local/squidGuard/squidGuard.conf

Blacklists

7. Now that SquidGuard is installed you will want to download some blacklists. The SquidGuard website provides a few options. Click on Blacklists link and download a few blacklists. I recommend going here http://cri.univ-tlse1.fr/blacklists/index_en.php and downloading the blacklists.tar.gz file from the top of the Descriptions section

Now you can move the blacklists to the SquidGuard db directory and extract them so they are ready to use.

cd ~/Downloads
sudo cp blacklists.tar.gz /usr/local/squidGuard/db/blacklists.tar.gz
cd /usr/local/squidGuard/db
sudo tar -xvf blacklists.tar.gz

Configuring SquidGuard

8. Now you are ready to configure SquidGuard you will want to open the configuration file with a text editor.
cd /usr/local/squidGuard/
ls
You should see a squidGuard.conf file. Copy the conf file to a backup and open it with a text editor
sudo cp squidGuard.conf squidGuard.conf.bak
sudo su
gedit squidGuard.conf &

If your squidGuard.conf file is janked i.e. blank, then you can copy the configuration directly from the SquidGuard website: http://www.squidguard.org/Doc/configure.html

Looking at your squidGuard.conf file in the text editor make sure that the lines beginning with dbhome and logdir point to the correct directory. For my install the dbhome and logdir lines read:
dbhome /usr/local/squidGuard/db
logdir /usr/local/squidGuard/log
So I changed the dbhome line to:
dbhome /usr/local/squidGuard/db/blacklists

Try to running squidGuard in an output to stderr mode:
squidGuard -d

I had errors showing on line 23 so I commented out lines 22 to 25 with # signs:
#rew dmz{
#          s@://admin/…
#          s@://foo.bar….
#}

Now try running squidGuard:
squidGuard -d

If squidGuard ran with no errors it is time to compile your Blacklists from text to DB with a -C all command
squidGuard -d -C all

{loadposition adposition7}I had additional errors caused by the Destination Classes area in the squidGuard.conf file. The dest adult block of code had the following lines that needed to have the “dest/” edited out, because they are not the correct directory paths following from the “/usr/local/squidGuard/db/blacklists” directory:

dest adult{
domainlist          dest/adult/domains
urllist                   dest/adult/urls
expressionlist    dest/adult/expressions
redirect               http://admin.foo.bar.de…
}

to

dest adult{
domainlist          adult/domains
urllist                   adult/urls
expressionlist    adult/expressions
redirect               http://google.com
}

I also edited the ACL block of code at the end of the config file. I commented out areas that I was not going to use, and focused on the default acl block of code, which I changed to pass only the not(!) adult sites (pass     !adult all):

acl {
#    admin {
#        pass     any
#    }
#
#    foo-clients within workhours {
#        pass     good !in-addr !adult any
#    } else {
#        pass any
#    }
#
#    bar-clients {
#        pass    local none
#    }
#
default {
pass     !adult all
#rewrite dmz
redirect http://google.com
}
}

9. After editing your config file try to compile your Blacklists from text to DB with a “-C all” command
squidGuard -d -C all

If there are no errors make sure the blacklists have correct ownership and group for Squid. You can check ownership of files and folders using the ls-l command. For Ubuntu the correct owner and group for Squid is “proxy”, in other distributions it is “squid”.
chown -R proxy:proxy /usr/local/squidGuard/db/blacklists

10. To finish the installation, add the following line to the squid.conf file in /etc/squid/squid.conf. I added the following line around line 1083 although you could add it anywhere, notice that it is directing the squidGuard program to the configuration file. If your squidGuard installation and configuration file is located in a different directory then adjust the paths in the line accordingly:

url_rewrite_program  /usr/local/bin/squidGuard  -c  /usr/local/squidGuard/squidGuard.conf

11. Now restart Squid or reload the Squid configuration file which is much faster.
service squid reload
or
pkill -9 squid
service squid start

12. In order to test if squidGuard configuration is working correctly and that Squid is passing web requests and checking them against the SquidGuard database. The SquidGuard website recommends running a dry-run test using the following command. You can substitute one of the blacklisted URLs from your blacklists instead of the “http://www.example.com” URL in the example. Also, If you do not have a “test.cfg” file, just remove the part of the line from, “-c … to … test.cfg” (see example below):

echo “http://www.example.com 10.0.0.1/ – – GET” | squidGuard -c /tmp/test.cfg -d
to
echo “http://www.blacklisted.com – – – GET” | squidGuard -d

After running the command above, if you see the following 3 messages in the output then squidGuard is functioning correctly:
– the redirected URL website address from the squidGuard.conf file
– “squidGuard ready for requests”,
– “squidguard stopped”

Now you can try using your web browser to see if it will block blacklisted domains and websites!

Note: If you are in a situation where you do not want to risk requesting blacklisted sites in your browser and having them not be filtered, then you can add one of your own entries in a blacklist, recompile the squidGuard blacklist database, and test to see if your manually entered website is blocked by squidGuard.

Squid in Ubuntu Overview

A proxy server is a very useful tool for a network. It is commonly used in computer networks to protect the network from attack, to filter nefarious web content and pages requested by local users, and to speed up the delivery of web pages and web content by caching (storing) commonly requested web pages, documents, and media. Proxy servers are typically implemented on private, local area networks, to filter, protect and cache content requested by users on that network, this is called “proxy” or “transparent proxy.” Proxy servers can also be implemented on the remote side “in-front-of” destination webservers in order to protect those servers by filtering requests, speeding up web page delivery, and caching frequently requested files, this is called “reverse proxy.”

Types of Proxy Servers

Squid is one of the most popular and most used proxy servers in the world. It is free to download, easy to install and it can be implemented on any distribution of Linux. Here are the steps to install and configure Squid on an Ubuntu distribution of Linux.

Steps to install and configure Squid

Open a terminal, and type in the following commands to install Squid
 sudo apt-get update
sudo apt-get install squid squid-common

Ways to start and stop Squid
 sudo service squid start (stop|restart|status)
sudo /usr/sbin/squid (launch program directly)
sudo pkill -9 squid

Navigate to the Squid folder to find the squid.conf configuration file
 cd /etc/squid
ls (you should see the squid.conf file)

Create a backup of the squid.conf file
 sudo cp squid.conf squid.conf.bak

For testing purposes open Firefox and set it to send web requests to the Squid Proxy Server (You will need to know your ip address)
 ifconfig (write down your inet address e.g. 192.168.1.100)
Open Firefox
Edit > Preferences, Advanced > Network Tab > Connection-Settings:
Manual Proxy Configuration:
HTTP Proxy: your IP address or loopback address 127.0.0.1,    Port: 3128
     Click Ok and Close
Now if you try and go to a website like google you should see an ERROR – Access Denied message from Squid (see bottom line). This means that Squid is working by actively denying the traffic.

Now we need to configure Squid to allow web traffic through the proxy server. Open squid.conf in your favorite text editor like gedit, nano, or vi
 sudo nano squid.conf
or
 sudo gedit squid.conf & (If gedit does not open from the terminal you can open it as root user)
sudo su
gedit squid.conf &

To switch out of root user
 su your-username (if you are root user the prompt is a “#” switch back to your user account privilege)

If you chose to open with squid.conf with gedit, then turn on line numbering (Edit > Preferences > View >Display Line Numbers)

Change the name of your Squid Proxy Server, around line 3399, change:
 #    TAG: visible_hostname
to
 visible_hostname YourNameProxyServer  

You can configure access rules for your Squid proxy server (lines 331 to 831 are for Access Control). Notice that on lines 606 to 630 the local networks and usable ports (services) are defined. Active configuration lines, are the lines that are not commented out, i.e. they do not start with  a # sign.

To re-enable web access uncomment line 676
 #http_access allow localnet
to
 http_access allow localnet

To verify the Web is now working, save your changes to the squid.conf file and restart your Squid server.
 service squid restart (or “sudo service squid restart” if you are no longer root)

Now resfresh your Firefox web browser and your homepage should be visible.

Now we can practice writing a custom ACL (access list) in the squid.conf file to block specific domains and websites. We can write our custom ACL at the end of the acl lines around line 631. From an empty line write the following lines to test domain blocking:
 acl blocked_websites dstdomain .msn.com .yahoo.com
http_access deny blocked_websites

Now restart your Squid server, and test to see if Squid denies access to your blocked domains/websites in Firefox.

Video Tutorials

In this series of videos, I go through the same process outlined above, to install and configure a Squid proxy server in Ubuntu .

In part 1, I install Squid in Ubuntu, start and stop it, backup the configuration file,
and configure Firefox to use Squid as a proxy server

In part 2, I discuss editing the configuration file

In part 3, I write a custom ACL in the squid.conf file