Install and Configure Squid in Ubuntu

Squid in Ubuntu Overview

A proxy server is a very useful tool for a network. It is commonly used in computer networks to protect the network from attack, to filter nefarious web content and pages requested by local users, and to speed up the delivery of web pages and web content by caching (storing) commonly requested web pages, documents, and media. Proxy servers are typically implemented on private, local area networks, to filter, protect and cache content requested by users on that network, this is called “proxy” or “transparent proxy.” Proxy servers can also be implemented on the remote side “in-front-of” destination webservers in order to protect those servers by filtering requests, speeding up web page delivery, and caching frequently requested files, this is called “reverse proxy.”

Types of Proxy Servers

Proxy Server The web browser on the client is configured to point to the proxy server’s IP address. The client can bypass the proxy server by removing or altering the proxy address configuration. An administrator could prevent this by creating a GPO in Active Directory that blocks access to the web browser settings. A proxy server can also function as a caching server.
Transparent Proxy Server The router sends all traffic on defined ports, to the transparent proxy server, this way clients cannot bypass the proxy server. A transparent proxy server can also function as a caching server.
Reverse Proxy Server (Caching) The reverse proxy server or cache server is placed in-front-of or prior-to the web server in order to speed up delivery of frequently requested pages and to protect the web server by creating a layer of separation and redundancy.

diagram of proxy server scenarios

Squid is one of the most popular and most used proxy servers in the world. It is free to download, easy to install and it can be implemented on any distribution of Linux. Here are the steps to install and configure Squid on an Ubuntu distribution of Linux.

Steps to install and configure Squid

Open a terminal, and type in the following commands to install Squid

sudo apt-get update
sudo apt-get install squid squid-common

Ways to start and stop Squid

sudo service squid start (stop|restart|status)
sudo /usr/sbin/squid (launch program directly)
sudo pkill -9 squid

Navigate to the Squid folder to find the squid.conf configuration file

cd /etc/squid
ls (you should see the squid.conf file)

Create a backup of the squid.conf file

sudo cp squid.conf squid.conf.bak

For testing purposes open Firefox and set it to send web requests to the Squid Proxy Server (You will need to know your ip address)

ifconfig (write down your inet address e.g. 192.168.1.100)
 

Open Firefox

Edit > Preferences, Advanced > Network Tab > Connection-Settings:

Manual Proxy Configuration:

HTTP Proxy: your IP address or loopback address 127.0.0.1,    Port: 3128
   Click Ok and Close

Now if you try and go to a website like google you should see an ERROR – Access Denied message from Squid (see bottom line). This means that Squid is working by actively denying the traffic.

Now we need to configure Squid to allow web traffic through the proxy server. Open squid.conf in your favorite text editor like gedit, nano, or vi

sudo nano squid.conf

or

sudo gedit squid.conf & (If gedit does not open from the terminal you can open it as root user)
sudo su
gedit squid.conf &

To switch out of root user

su your-username (if you are root user the prompt is a "#" switch back to your user account privilege)

If you chose to open with squid.conf with gedit, then turn on line numbering (Edit > Preferences > View >Display Line Numbers)

Change the name of your Squid Proxy Server, around line 3399, change.

#    TAG: visible_hostname

to

 visible_hostname YourNameProxyServer  

You can configure access rules for your Squid proxy server (lines 331 to 831 are for Access Control). Notice that on lines 606 to 630 the local networks and usable ports (services) are defined. Active configuration lines, are the lines that are not commented out, i.e. they do not start with  a # sign.

To re-enable web access uncomment line 676

#http_access allow localnet

to

http_access allow localnet

To verify the Web is now working, save your changes to the squid.conf file and restart your Squid server.

sudo service squid restart

Now resfresh your Firefox web browser and your homepage should be visible.

Now we can practice writing a custom ACL (access list) in the squid.conf file to block specific domains and websites. We can write our custom ACL at the end of the acl lines around line 631. From an empty line write the following lines to test domain blocking.

acl blocked_websites dstdomain .msn.com .yahoo.com
http_access deny blocked_websites

Now restart your Squid server, and test to see if Squid denies access to your blocked domains/websites in Firefox.

Video Tutorials

In this series of videos, I go through the same process outlined above, to install and configure a Squid proxy server in Ubuntu .

In part 1, I install Squid in Ubuntu, start and stop it, backup the configuration
file, and configure Firefox to use Squid as a proxy server

In part 2, I discuss editing the configuration file

In part 3, I write a custom ACL in the squid.conf file

Install & Configure SquidGuard in Ubuntu

Install and Configure SquidGuard Overview

It is very useful to be able to block users on your network from accessing millions of websites with nefarious content. A great way to accomplish this is with a proxy server like Squid. Squid is a free and powerful proxy server that is capable of blocking users from accessing web content. A great way of enhancing Squid’s ability to block unwanted websites, domains and IP addresses is to install SquidGuard. SquidGuard is an add-on program for the Squid proxy server (see my previous article on Squid), that’s main purpose is to block unwanted web traffic. SquidGuard works with databases of blacklists to block, filter, and redirect requested URLs and domains. You manually download and add blacklist files to SquidGuard and compile them into the SquidGuard database, then Squid can redirect web requests, checking them against SquidGuard’s database of blacklisted websites, domains and IP addresses. It seems like this process would slow down a network, but SquidGuard is an extremely fast web content filter with the ability to check web requests against millions of blacklisted sites in a matter of seconds. There is great information about SquidGuard’s capabilities on the SquidGuard website, including links to download the program, well written installation and configuration instructions, and links to websites that maintain blacklists.

Steps to manually install SquidGuard in Ubuntu

You can download and install SquidGuard using a package manager program like apt-get or yum, or even a graphical software installer tool like the software center program. Instead, I chose to outline the steps involved in manually downloading and installing SquidGuard.

1. Download the current stable version of SquidGuard at http://www.squidguard.org and save it to your downloads folder.

2. Download the Berkeley DB from Oracle at http://oracle.com. Download version 4.8.30.NC.tar.gz with no encryption and save it to your Downloads folder

3. Open a terminal and navigate to the directory where you downloaded SquidGuard and the BerkeleyDB. You should see the tar.gz files
cd ~/Downloads
ls

4. Decompress the tar.gz files (substitute the file names for the versions you downloaded)
tar -xvzf squidGuard-1.5-beta.tar.gz
tar -xvzf db-4.8.30.NC.tar.gz
ls

You should see two folders one for squidGuard and one for BerkeleyDB (e.g. squidGuard-1.5, and db-4.8.30)

5. Install the Berkeley DB first, since SquidGuard requires it for installation. By default, the Berkeley DB will install itself to a directory in /usr/local/ in a folder named BerkeleyDB.4.8 you will need this information when preparing SquidGuard for installation.
cd db-4.8.30
cd build_unix
../dist/configure
make
sudo make install

6. Install SquidGuard by navigating to the extracted SquidGuard folder and then during the configure process you will pass the configure script the location of the Berkeley DB directory and correctly change the squiduser to ‘proxy’ for Ubuntu. The squiduser and group is typically “squid” in other Linux distributions like Fedora.
cd ~/Downloads/squidGuard-1.5
./configure –with-db=/usr/local/BerkeleyDB.4.8 –with-squiduser=proxy
make
sudo make install

You should get a message that the initial SquidGuard configuration is complete. Congratulation, SquidGuard is successfully installed! Make a note of the directory locations of the SquidGurad db, log, and conf files:
/usr/local/squidGuard/db
/usr/local/squidGuard/log
/usr/local/squidGuard/squidGuard.conf

Blacklists

7. Now that SquidGuard is installed you will want to download some blacklists. The SquidGuard website provides a few options. Click on Blacklists link and download a few blacklists. I recommend going here http://cri.univ-tlse1.fr/blacklists/index_en.php and downloading the blacklists.tar.gz file from the top of the Descriptions section

Now you can move the blacklists to the SquidGuard db directory and extract them so they are ready to use.

cd ~/Downloads
sudo cp blacklists.tar.gz /usr/local/squidGuard/db/blacklists.tar.gz
cd /usr/local/squidGuard/db
sudo tar -xvf blacklists.tar.gz

Configuring SquidGuard

8. Now you are ready to configure SquidGuard you will want to open the configuration file with a text editor.
cd /usr/local/squidGuard/
ls

You should see a squidGuard.conf file. Copy the conf file to a backup and open it with a text editor
sudo cp squidGuard.conf squidGuard.conf.bak
sudo su
gedit squidGuard.conf &

If your squidGuard.conf file is janked i.e. blank, then you can copy the configuration directly from the SquidGuard website: http://www.squidguard.org/Doc/configure.html

Looking at your squidGuard.conf file in the text editor make sure that the lines beginning with dbhome and logdir point to the correct directory. For my install the dbhome and logdir lines read:
dbhome /usr/local/squidGuard/db
logdir /usr/local/squidGuard/log

So I changed the dbhome line to:
dbhome /usr/local/squidGuard/db/blacklists

Try to running squidGuard in an output to stderr mode:
squidGuard -d

I had errors showing on line 23 so I commented out lines 22 to 25 with # signs:
#rew dmz{
#          s@://admin/…
#          s@://foo.bar….
#}

Now try running squidGuard:
squidGuard -d

If squidGuard ran with no errors it is time to compile your Blacklists from text to DB with a -C all command
squidGuard -d -C all

{loadposition adposition7}I had additional errors caused by the Destination Classes area in the squidGuard.conf file. The dest adult block of code had the following lines that needed to have the “dest/” edited out, because they are not the correct directory paths following from the “/usr/local/squidGuard/db/blacklists” directory:

dest adult{
domainlist          dest/adult/domains
urllist                   dest/adult/urls
expressionlist    dest/adult/expressions
redirect               http://admin.foo.bar.de…
}

to

dest adult{
domainlist          adult/domains
urllist                   adult/urls
expressionlist    adult/expressions
redirect               http://google.com
}

I also edited the ACL block of code at the end of the config file. I commented out areas that I was not going to use, and focused on the default acl block of code, which I changed to pass only the not(!) adult sites (pass     !adult all):

acl {
#    admin {
#        pass     any
#    }
#
#    foo-clients within workhours {
#        pass     good !in-addr !adult any
#    } else {
#        pass any
#    }
#
#    bar-clients {
#        pass    local none
#    }
#
default {
pass     !adult all
#rewrite dmz
redirect http://google.com
}
}

9. After editing your config file try to compile your Blacklists from text to DB with a “-C all” command
squidGuard -d -C all

If there are no errors make sure the blacklists have correct ownership and group for Squid. You can check ownership of files and folders using the ls-l command. For Ubuntu the correct owner and group for Squid is “proxy”, in other distributions it is “squid”.
chown -R proxy:proxy /usr/local/squidGuard/db/blacklists

10. To finish the installation, add the following line to the squid.conf file in /etc/squid/squid.conf. I added the following line around line 1083 although you could add it anywhere, notice that it is directing the squidGuard program to the configuration file. If your squidGuard installation and configuration file is located in a different directory then adjust the paths in the line accordingly:

url_rewrite_program  /usr/local/bin/squidGuard  -c  /usr/local/squidGuard/squidGuard.conf

11. Now restart Squid or reload the Squid configuration file which is much faster.
service squid reload
or
pkill -9 squid
service squid start

12. In order to test if squidGuard configuration is working correctly and that Squid is passing web requests and checking them against the SquidGuard database. The SquidGuard website recommends running a dry-run test using the following command. You can substitute one of the blacklisted URLs from your blacklists instead of the “http://www.example.com” URL in the example. Also, If you do not have a “test.cfg” file, just remove the part of the line from, “-c … to … test.cfg” (see example below):

echo “http://www.example.com 10.0.0.1/ – – GET” | squidGuard -c /tmp/test.cfg -d
to
echo “http://www.blacklisted.com – – – GET” | squidGuard -d

After running the command above, if you see the following 3 messages in the output then squidGuard is functioning correctly:
– the redirected URL website address from the squidGuard.conf file
– “squidGuard ready for requests”,
– “squidguard stopped”

Now you can try using your web browser to see if it will block blacklisted domains and websites!

Note: If you are in a situation where you do not want to risk requesting blacklisted sites in your browser and having them not be filtered, then you can add one of your own entries in a blacklist, recompile the squidGuard blacklist database, and test to see if your manually entered website is blocked by squidGuard.

Setup Remote Desktop Sharing in Ubuntu with VNC

Video Tutorial – VNC

In this tutorial, I set up remote desktop sharing in Ubuntu and then
connect to Ubuntu from a Windows 7 computer using the Real VNC Client

Video Tutorial – TeamViewer

In this tutorial, I setup a remote desktop connection from Windows 7 to Ubuntu 11.10 using TeamViewer

Upgrade & Personalize Ubuntu 10

Overview

I recently purchased the magazine Linux Identity Office Ubuntu 10.10 Maverick Meercat. It came with 2 dvds: one a 2-sided bootable Ubuntu install dvd with 32 bit on one side and 64 bit on the other. The other dvd is a Bootable Live CD. You can order the magazine by going to the website here:
http://www.linuxidentity.com/us/index.php?name=News

See that specific issue’s table of contents and one viewable article here:
http://www.linuxidentity.com/us/index.php?name=News&file=article&sid=5048

Since I have a new install of Ubuntu 10.10 on a laptop I decided to follow their article on things a user can do after a fresh install of Ubuntu. Here are some of the things they recommend doing:

Enable all of the available repositories so you can have access to installing a larger variety of applications, drivers and utilities

  1. Under System select Administration
  2. Checkmark “Software Sources” and press Close
  3. Now open System > Administration > Software Sources
  4. If they aren’t already check-marked you can check off “Proprietary drivers … (restricted)” and “Software restricted … (multiverse)”
  5. Click on the Other Software Tab and if they aren’t already check-marked check off the “Canonical Partners” and “Third-Party Software Developers” packages as well.
  6. Now you will have access to more installable applications and drivers!

Run the best drivers for your graphics card and wireless card

  1. System > Administration > Additional Drivers
  2. Ubuntu now searches to see if there are better drivers for your computer and lets you know if you are running proprietary drivers.
  3. On my laptop I discovered I am running a Broadcom proprietary driver for my wireless card. In the past, getting a wireless driver so easily installed on Linux was not always possible.

Enable desktop effects and Compiz

  1. System > Preferences > Appearance
  2. Click on “Visual Effects” and you will see three levels of desktop effects. Choose which one is right for you based on the power of your graphics card.
  3. My laptop since it is a little older is set to the “Normal” setting
  4. If you want some really cool desktop effects and your computer has a decent graphics card you may want to install the “CompizConfig Settings Manager”.
  5. To install Compiz open a Terminal (Applications > Accessories > Teminal) and type
    sudo apt-get install compizconfig-settings-manager
    (You will need to give your admin password and key in “y” + enter when prompted to execute the install)
  6. Now go to System > Preferences > CompizConfig Settings Manager and have FUN! Make sure you try “Wobbly Windows” under “Effects”and also try the “Desktop Cube”. In the CompizConfig Settings Manager Click on the “Desktop” section and then checkmark “Desktop Cube” and “Rotate Cube.” Now holding down the Ctrl + Alt keys and press the “Right Arrow” and “Left Arrow” keys … or holding down the Ctrl + Alt keys click and drag …. Oooooo!

Install a Docking Application

  1. Three docking applications are recommended: Avant Window Navigator, GNOME Do, and Cairo.
  2. To install AWN Avant Window Navigator. Open a terminal and type:
    sudo apt-get install avant-window-navigator
  3. Once the install is complete you can start the program by going to:Applications > Accessories > Avant Window Navigator
  4. You can Right+Click on the dock to change the preferences

Upgrade & Personalize Ubuntu 11

Overview

Ubuntu 11.10 codename Oneiric Ocelot has a completely different look and feel compared earlier releases off Ubuntu. Once you have installed this latest version of Ubuntu you may find yourself lost and fumbling around in the dark with the new Unity desktop. Well it is normal for “the new,” to feel a little uncomfortable, but that doesn’t mean that is necessarily bad.

Once you have your new install of Ubuntu 11.10 up and running, you will want to run updates especiall security updates, as well as upgrade and customize your system, like changing the desktop, customizing your menus, installing media players and codecs, etc..

 

Here are three websites that will help you with this process:
http://www.techdrivein.com/2011/10/15-things-i-did-after-installing-new.html ,
http://www.omgubuntu.co.uk/2011/0/gnome-shell-ubuntu-11-10-guide/ ,
http://maketecheasier.com/install-custom-gnome-shell-themes/2011/09/27

 

New Install “To Do List!”
Open a Terminal shell and type in the following commands:
Command Explanation
sudo apt-get update Download a list of updated program packages
sudo apt-get upgrade Download upgraded program packages
sudo apt-get install ubuntu-restricted-extras Install support for media formats mp3, avi, Flash
sudo apt-get install compizconfig-settings-manager Install a graphical desktop settings manager
sudo apt-get install gnome Install the Gnome desktop
   
   
   
   

 

 

Install Ubuntu 11.10

Overview

Ubuntu is one of the most popular distributions of Linux today. Based on the Debian Linux distribution, Ubuntu is designed for the desktop, home user although there are other Ubuntu variations, like Ubuntu Server that does not install a graphical user interface, Edubuntu designed for educational applications, Kubuntu designed for the KDE desktop, Ubuntu Studio designed as a professional video and audio editing workstation, and Mythbuntu designed as a MythTV home theater PC.

The Ubuntu operating system is freely distributed and is supported by Canonical LTD. which also sells technical support and services.

Installing Ubuntu

As of now, the current version of Ubuntu is 11.10, named “Oneiric Ocelot,” it is the second release of Ubuntu to use the Unity desktop written on top of the GNOME 3.x desktop. You can download Ubuntu for free at the Ubuntu website: http://www.ubuntu.com . The download page offers different options for installing Ubuntu including downloading a burnable iso bootable image, installing Ubuntu alongside Windows in a dual boot system, or creating a USB bootable thumb drive.

In the following tutorial, I download the Ubuntu 32bit iso file and install it in a VMware virtual machine on my Windows 7 laptop.

 

Install and Configure Samba in Ubuntu

Overview

Samba is a file and print server that you can install on a Linux distribution like Ubuntu. It is useful for sharing files, folders, and printers with Windows users over a local network. In this lab, the goal is to install Samba in Linux, set up a shared folder and text file, and from a different computer running Windows workgroup, connect to the Samba share and access the shared folder and file. The lab is complete when you have open the shared file, added some text like “hello” and saved the file.

Installation and Configuration

To install Samba in Ubuntu, I followed the instructions on this website: https://help.ubuntu.com/10.04/serverguide/C/samba-fileserver.html

Open a terminal and put in the following commands (in the examples, “sudo” is used for root privileges):

  1. Update your repositories:
        sudo apt-get update
  2. Install with apt:
        sudo apt-get install samba
  3. Open the samba configuration file smb.conf. In this example I use the gedit text editor and I background the process with “&” so I can continue to use my terminal without having to close the text file first:
        sudo gedit /etc/samba/smb.conf &
  4. In the Global Settings section of the configuration file, change or verify the following lines:
    workgroup = WORKGROUP
    and in the Authentication section, change or verify the following lines:
    security = user
    I chose “WORKGROUP” because it is the default workgroup name in Windows.
  5. At the bottom on the configuration file add the following lines then save the file:
      [share]
    comment = Ubuntu File Server Share
    path = /srv/samba/share
    browsable = yes
    guest ok = yes
    read only = no
    create mask = 0755
    The path is the path to the shared folder, browsable allows the share to be visible to Windows explorer users, guest ok allows a user to access the share without supplying a password, create mask sets the permissions for everything created within the share.
  6. Now you need to create the directory folder to use as a share and change the owner and group:
        sudo mkdir -p /srv/samba/share
    sudo chown nobody.nogroup /srv/samba/share
  7. You can also create a sample text file in your shared folder to practice sharing to windows:
        sudo touch /srv/samba/share/test.txt
  8. Now restart samba:
        sudo restart smbd
        sudo restart nmbd (If you have nmbd failure, I found a temporary hack, see the bottom of this page)
  9. Now restart smbd and nmbd and it should work (after restarting if nmbd still does not work try restarting Ubuntu):
        sudo restart smbd
    sudo restart nmbd

  10. Now that you have Samba up and running go to another computer on your network that is running Windows and look for the share. Try the following steps and methods for locating your network share on Windows:
    1. open a folder (File Explorer) and type the IP address of the Samba server computer in the address bar (e.g.: \\192.168.1.100) and hit enter. If you are asked for a username and password, skip ahead to number 5.
    2. or, go to the Network and Sharing Center
    3. on the left hand side of the window click on “View computers and devices”
    4. if you are prompted by a drop down box, turn on “Network Discovery and File Sharing” for all public networks
    5. I had to put in my username and password like this:
      <username>@<computer-name>
      In my case it was:
      dan@dan-VirtualBox
      If you are unsure, open a terminal in you Linux machine and the prompt will be your user name + @ + your computer name.
    6. once authenticated you should have access to the file shares and shared printers that you created and configured in Samba.
    In this image, you can see the “Network and Sharing Center” window, the “View Computers and devices”
    window, as well as my Samba share “DAN-VIRTUALBOX”, and a window with a shared text file


Video Tutorial

 


***Troubleshooting note:

  1. If you notice that nmbd is failing… This bug seems to be a recent occurrence in Ubuntu 10. Here is a temporary hack that I found on a separate website blog. Open the nmbd.conf file in gedit:
        sudo gedit /etc/init/nmbd.conf &
    Comment out the following lines in the nmbd.conf file by adding a “#” at the beginning of the line like this:
        #          NMBD_DISABLED=`testparm -s –paramenter-name=’disable netbios’  2>/dev/null`
    #          [   “x$NMBD_DISABLED” = xYes  ]  &&  {  stop; exit 0;  }