Tuesday, May 22, 2012

How to integrate Squid Proxy Server with Active Directory

In this blog we will see hot to setup Squid Proxy which will seamlessly integrate with Active Directory using Kerberos, NTLM and basic authentication for clients not authenticated via Kerberos or NTLM.
Example Environment
 Domain= linuxhowto.com
 Subnet =
 Proxy Server
 IP =
 HOSTNAME = squidproxy.linuxhowto.com
 Kerberos computer name = SQUIDPROXY-K
 Windows Server 1
 IP =
 HOSTNAME = dc1.linuxhowto.com
 Windows Server 2
 IP =
 HOSTNAME = dc2.linuxhowto.com


Client Windows Computers need to have "Enable Integrated Windows Authentication" ticked in Internet Optionss ===>Advanced Settings.

DNS Configuration

On the Windows DNS server add a new A record entry for the proxy server's hostname and ensure a corresponding PTR (reverse DNS) entry is also created and works.

Check that the proxy is using the Windows DNS Server for name resolution and update /etc/resolv.conf accordingly.

Edit the file according to your network.
domain linuxhowto.com
search linuxhowto.com

Ping a internal and external hostname to ensure DNS is operating.
ping dc1.linuxhowto.com -c 4 && ping google.com -c 4
PING dc1.linuxhowto.com ( 56(84) bytes of data.
64 bytes from dc1.linuxhowto.com ( icmp_req=1 ttl=64 time=0.028 ms
64 bytes from dc1.linuxhowto.com ( icmp_req=2 ttl=64 time=0.017 ms
64 bytes from dc1.linuxhowto.com ( icmp_req=3 ttl=64 time=0.013 ms
64 bytes from dc1.linuxhowto.com ( icmp_req=4 ttl=64 time=0.009 ms

--- dc1.linuxhowto.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2997ms
rtt min/avg/max/mdev = 0.009/0.016/0.028/0.008 ms
PING google.com ( 56(84) bytes of data.
64 bytes from syd01s13-in-f20.1e100.net ( icmp_req=1 ttl=55 time=40.4 ms
64 bytes from syd01s13-in-f20.1e100.net ( icmp_req=2 ttl=55 time=42.2 ms
64 bytes from syd01s13-in-f20.1e100.net ( icmp_req=3 ttl=55 time=41.2 ms
64 bytes from syd01s13-in-f20.1e100.net ( icmp_req=4 ttl=55 time=42.0 ms

--- google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 40.497/41.511/42.248/0.685 ms

Check you can reverse lookup the Windows Server and the local proxy ip from the Windows DNS.
dig -x
; <<>> DiG 9.7.3 <<>> -x
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23429
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;         IN      PTR

;; ANSWER SECTION:  1200    IN      PTR     dc1.linuxhowto.com.

;; Query time: 3 msec
;; WHEN: Thu Dec 29 14:09:40 2011
;; MSG SIZE  rcvd: 81

dig -x
; <<>> DiG 9.7.3 <<>> -x
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23429
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;         IN      PTR

;; ANSWER SECTION:  1200    IN      PTR     squidproxy.linuxhowto.com.

;; Query time: 3 msec
;; WHEN: Thu Dec 29 14:14:33 2011
;; MSG SIZE  rcvd: 85

 Warning: If either lookup fails do not proceed until fixed or authentication may fail.

NTP Configuration

Because Kerberos needs to have the time synchronized with Windows Domain Controllers for authentication we configure the proxy to obtain time from them. 

vi /etc/ntp.conf

Locate the following section and update the ntp servers as required. If you have more than one Domain Controller or NTP Server you may add multiple lines.
# You do need to talk to an NTP server or two (or three).
#server ntp.your-provider.example
server dc1.linuxhowto.com
server dc2.linuxhowto.com

Restart and test NTP.
service ntp restart

Run the following ntpq command, you should see output that refers to the Domain Controllers and other NTP Servers which are processed in the order that they appear in the conf file.
ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
 dc1.example.loc .LOCL.           1 u   32 1024  377    0.463    1.874   9.718
 dc2.example.loc      2 u  202 1024  377    1.032  -20.487   9.975

Install and Configure Kerberos

Install Kerberos packages
apt-get install krb5-user libkrb53
 Note: Just accept the defaults presented to any debconf dialog's presented as we are overwriting them.

Setup Kerberos.
cp /etc/krb5.conf /etc/krb5.conf.default
cat /dev/null > /etc/krb5.conf
vi /etc/krb5.conf

Edit the file replacing the variables with the client's domain and server.
 Note: If you only have 1 Domain Controller remove the additional 'kdc' entry from the '[realms]' section, or add any additional DC's
 Warning: Depending on your Domain Controller's OS Version uncomment the relevant Windows 200X section and comment out the opposing section.
    default_realm = LINUXHOWTO.COM
    dns_lookup_kdc = no
    dns_lookup_realm = no
    ticket_lifetime = 24h
    default_keytab_name = /etc/squid3/PROXY.keytab

; for Windows 2003
    default_tgs_enctypes = rc4-hmac des-cbc-crc des-cbc-md5
    default_tkt_enctypes = rc4-hmac des-cbc-crc des-cbc-md5
    permitted_enctypes = rc4-hmac des-cbc-crc des-cbc-md5

; for Windows 2008 with AES
;    default_tgs_enctypes = aes256-cts-hmac-sha1-96 rc4-hmac des-cbc-crc des-cbc-md5
;    default_tkt_enctypes = aes256-cts-hmac-sha1-96 rc4-hmac des-cbc-crc des-cbc-md5
;    permitted_enctypes = aes256-cts-hmac-sha1-96 rc4-hmac des-cbc-crc des-cbc-md5

        kdc = dc1.linuxhowto.com
        kdc = dc2.linuxhowto.com
        admin_server = dc1.linuxhowto.com
        default_domain = linuxhowto.com

    .linuxhowto.com = LINUXHOWTO.COM
    linuxhowto.com = LINUXHOWTO.COM

Install Squid 3

We install squid now as we need the squid3 directories available. Squid configuration takes places after authentication is configured.
apt-get install squid3 ldap-utils


Kerberos utilises msktutil an Active Directory keytab manager (I presume the name is abbreviated for "Microsoft Keytab Utility"). We need to install some packages that msktutil requires.
apt-get install libsasl2-modules-gssapi-mit libsasl2-modules

To make the following code easier to copy and paste run the following command, subsitute the MSKTARCH variable with i386 if necessary.
export MSKTARCH=amd64

Then obtain the msktutil package and install it.
wget -O /var/cache/apt/archives/msktutil_0.4-2_$MSKTARCH.deb "http://fuhm.net/software/msktutil/releases/msktutil_0.4-2_$MSKTARCH.deb"
dpkg -i /var/cache/apt/archives/msktutil_0.4-2_$MSKTARCH.deb

Initiate a kerberos session to the server with administrator permissions to add objects to AD, update the username where necessary. msktutil will use it to create our kerberos computer object in Active directory.
kinit administrator
 Password for administrator@LINUXHOWTO.COM:

It should return without errors. You can see if you successfully obtained a ticket with: 
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: administrator@LINUXHOWTO.COM
 Valid starting     Expires            Service principal
01/09/12 09:01:49  01/09/12 19:01:53  krbtgt/LINUXHOWTO.COM@LINUXHOWTO.COM
        renew until 01/10/12 09:01:49

Now we configure the proxy's kerberos computer account and service principle by running msktutil (remember to update the highlighted values with yours).
 Warning: There are 2 important caveats in regard to the msktutils --computer-name argument.
--computer-name cannot be longer than 15 characters due to netbios name limitations. See this link and this link for further information.
--computer-name must be different from the proxy's hostname so computer account password updates for NTLM and Kerberos do not conflict, see this link for further
information. This guide uses -k appended to the hostname.

Execute the msktutil command as follows:
msktutil -c -b "CN=COMPUTERS" -s HTTP/squidproxy.linuxhowto.com -k /etc/squid3/PROXY.keytab \
--computer-name SQUIDPROXY-K --upn HTTP/squidproxy.linuxhowto.com --server dc1.linuxhowto.com --verbose
 Warning: If you are using a Server 2008 domain then add --enctypes 28 at the end of the command
msktutil -c -b "CN=COMPUTERS" -s HTTP/squidproxy.linuxhowto.com -k /etc/squid3/PROXY.keytab \
--computer-name SQUIDPROXY-K --upn HTTP/squidproxy.linuxhowto.com --server dc1.linuxhowto.com --verbose --enctypes 28
 Tip: an example of adding the proxy to an OU would be -b ou=MEMBER SERVERS,ou=EXAMPLE

Pay attention to the output of the command to ensure success, because we are using --verbose output you should review it carefully.

Set the permissions on the keytab so squid can read it.
chgrp proxy /etc/squid3/PROXY.keytab
chmod g+r /etc/squid3/PROXY.keytab

Destroy the administrator credentials used to create the account.

On the Windows Server reset the Computer Account in AD by right clicking on the SQUIDPROXY-K Computer object and select "Reset Account", then run msktutil as follows to ensure the keytab is updated as expected and that the keytab is being sourced by msktutil from /etc/krb5.conf correctly. This is not completely necessary but is useful to ensure msktutil works as expected.

Then run the following:
msktutil --auto-update --verbose --computer-name squidproxy-k
 Note: Even though the account was added in capital letters, the --auto-update in msktutil requires the
--computer-name to be lower case.

If the keytab is not found try adding -k /etc/squid3/PROXY.keytab to the command to see if it works and then troubleshoot until resolved or users will not be able to
authenticate with Squid.

Add the following to cron so it can automatically updates the computer account in active directory when it expires (typically 30 days). I pipe it through logger so I can see any errors in syslog if necessary. As stated msktutil uses the default /etc/krb5.conf file for its paramaters so be aware of that if you decide to make any changes in it.

The SHELL and PATH variables are there to ensure cron runs properly, change this if you know what your doing.
crontab -e
# m h  dom mon dow   command
00 4  *   *   *     msktutil --auto-update --verbose --computer-name squidproxy-k | logger -t msktutil

Add the following configuration to /etc/default/squid3 so squid knows where to find the kerberos keytab.
vi /etc/default/squid3
export KRB5_KTNAME


Install Samba and Winbind
apt-get install samba winbind samba-common-bin

Stop the samba and winbind daemons

/etc/init.d/winbind stop && /etc/init.d/samba stop

Copy the default smb.conf out of the way and edit the smb.conf
cp /etc/samba/smb.conf /etc/samba/smb.conf.default
cat /dev/null > /etc/samba/smb.conf
vi /etc/samba/smb.conf
local master = no
workgroup = EXAMPLE
security = ads

winbind uid = 10000-20000
winbind gid = 10000-20000
winbind use default domain = yes
winbind enum users = yes
winbind enum groups = yes

load printers = no
printing = bsd
printcap name = /dev/null
disable spoolss = yes

Now join the proxy to the domain.

net ads join -U Administrator
Enter Administrator's password:
Using short domain name -- EXAMPLE
Joined 'SQUIDPROXY' to realm 'linuxhowto.com'

Start samba and winbind and test acces to the domain
/etc/init.d/ samba start && /etc/init.d/winbind start
wbinfo -t
checking the trust secret for domain EXAMPLE via RPC calls succeeded
wbinfo -a EXAMPLE\\testuser%'password'
plaintext password authentication succeeded
challenge/response password authentication succeeded

Set Permissions so the proxy user account can read /var/run/samba/winbindd_privileged
gpasswd -a proxy winbindd_priv

append the following to cron to regularly change the computer account password - internal note: Need to research if Samba does this automatically.
crontab -e
05  4  *   *   *     net rpc changetrustpw -d 1 | logger -t changetrustpw

In order to use basic authentication by way of LDAP we need to create an account with which to access Active Directory. 
 In Active Directory create a user called "Squid Proxy" with the logon name squid@linuxhowto.com. 
 Ensure the following is true when creating the account. 
 User must change password at next logon Unticked
 User cannot change password Ticked
 Password never expires Ticked
 Account is disabled Unticked

Create a password file used by squid for ldap access and secure the file permissions (substitute the word "squidpass" below with your password).
echo 'squidpass' > /etc/squid3/ldappass.txt
chmod o-r /etc/squid3/ldappass.txt
chgrp proxy /etc/squid3/ldappass.txt

Access Groups

Authorisation to use the internet is managed via Security Groups in Active Directory.

By default the squid account will not be able to query the "memberOf" attribute in AD. Select the top level of your active directory domain in Active Directory Users and Computers, Right click on it and choose properties, Security Tab, Add the squid user and give it read permissions (should happen by default) and allow it to read "This Object and all child objects" (Server 2003) or "This Object and all descendant objects" (Server 2008) by going into Advanced options.
 Tip: If you do not see the Security Tab on the domain properties window, select View and tick Advanced features from the AD Users and Computers MMC

Create the following Security Groups and descriptions in AD and add users to the relevant groups. I suggest adding all your users to Internet Users Standard and then increasing or decreasing their access level by adding them to additional groups. The order of access is from least access to highest. So for example, if a user was a member of Blocked, Standard and Anonymous, Blocked takes priority and they would have no internet access.
 Internet Users Blocked
 Description: Members of this group have no internet access
 Internet Users Restricted
 Description: Members of this group can access the internet allowed sites only
 Internet Users Standard
 Description: Members of this group can access the internet except for blocked sites
 Internet Users Exception
 Description: Members of this group can access the internet with exceptions to blocked sites
 Internet Users Full
 Description: Members of this group have full internet access
 Internet Users Anonymous
 Description: Members of this group have full internet access and no access is logged

Create the associated files on the proxy. Squid will use these to lookup group membership for users.
echo 'Internet Users Blocked' > /etc/squid3/blocked_access.txt
echo 'Internet Users Restricted' > /etc/squid3/restricted_access.txt
echo 'Internet Users Standard' > /etc/squid3/standard_access.txt
echo 'Internet Users Exception' > /etc/squid3/exception_access.txt
echo 'Internet Users Full' > /etc/squid3/full_access.txt
echo 'Internet Users Anonymous' > /etc/squid3/anonymous_access.txt
 Note: After making changes to group membership squid needs to be reloaded on the Proxy

/etc/init.d/squid3 reload

Configure Squid
Install negotiate_wrapper

Firstly we need to install negotiate_wrapper. Install the necessary build tools.
apt-get install build-essential linux-headers-$(uname -r)

Then compile and install.
cd /usr/local/src/
wget "http://downloads.sourceforge.net/project/squidkerbauth/negotiate_wrapper/negotiate_wrapper-1.0.1/negotiate_wrapper-1.0.1.tar.gz"
tar -xvzf negotiate_wrapper-1.0.1.tar.gz
cd negotiate_wrapper-1.0.1/
make install

We then setup squid and it's associated config files.
cp /etc/squid3/squid.conf /etc/squid3/squid.conf.default
cat /dev/null > /etc/squid3/squid.conf
vi /etc/squid3/squid.conf

 Note: Update the cache manager variable with the emails address of the person in charge of the proxy

Study and update the following text carefully, replacing the example content with your networks configuration - if you get something wrong your proxy will not work.
 Note: Although it may work, wrapping long lines with " \" in squid.conf is not a valid - don't do it.
### /etc/squid3/squid.conf Configuration File ####

### cache manager
cache_mgr cache@example.com

### negotiate kerberos and ntlm authentication
auth_param negotiate program /usr/local/bin/negotiate_wrapper -d --ntlm /usr/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp --domain=EXAMPLE --kerberos /usr/lib/squid3/squid_kerb_auth -d -s GSS_C_NO_NAME
auth_param negotiate children 10
auth_param negotiate keep_alive off

### pure ntlm authentication
auth_param ntlm program /usr/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp --domain=EXAMPLE
auth_param ntlm children 10
auth_param ntlm keep_alive off

### provide basic authentication via ldap for clients not authenticated via kerberos/ntlm
auth_param basic program /usr/lib/squid3/squid_ldap_auth -R -b "dc=example,dc=local" -D squid@linuxhowto.com -W /etc/squid3/ldappass.txt -f sAMAccountName=%s -h dc1.linuxhowto.com
auth_param basic children 10
auth_param basic realm Internet Proxy
auth_param basic credentialsttl 1 minute

### ldap authorisation
external_acl_type memberof %LOGIN /usr/lib/squid3/squid_ldap_group -R -K -b "dc=example,dc=local" -D squid@linuxhowto.com -W /etc/squid3/ldappass.txt -f "(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=%g,ou=Security Groups,ou=MyBusiness,dc=example,dc=local))" -h dc1.linuxhowto.com

### acl for proxy auth and ldap authorizations
acl auth proxy_auth REQUIRED
#   aclname             acltype  typename activedirectorygroup
acl BlockedAccess       external memberof "/etc/squid3/blocked_access.txt"
acl RestrictedAccess    external memberof "/etc/squid3/restricted_access.txt"
acl StandardAccess      external memberof "/etc/squid3/standard_access.txt"
acl ExceptionAccess     external memberof "/etc/squid3/exception_access.txt"
acl FullAccess          external memberof "/etc/squid3/full_access.txt"
acl AnonymousAccess     external memberof "/etc/squid3/anonymous_access.txt"
acl allowedsites        dstdomain "/etc/squid3/allowedsites.txt"
acl blockedsites        dstdomain "/etc/squid3/blockedsites.txt"
acl exceptedsites       dstdomain "/etc/squid3/exceptedsites.txt"
acl prioritysites       dstdomain "/etc/squid3/prioritysites.txt"

### squid defaults
acl manager proto cache_object
acl localhost src ::1
acl to_localhost dst ::1
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost

### http_access rules
# allow unrestricted access to prioritysites
http_access allow prioritysites
# enforce authentication, order of rules is important for authorization levels
http_access deny !auth
# prevent access to basic auth prompt for BlockedAccess users
http_access deny BlockedAccess all
http_access allow allowedsites
http_access deny RestrictedAccess all
http_access allow AnonymousAccess auth
http_access allow FullAccess auth
http_access allow ExceptionAccess exceptedsites auth
http_access deny blockedsites
http_access allow StandardAccess auth
http_access deny all

### logging
# don't log allowedsites, prioritysites, AnonymousAccess
access_log /var/log/squid3/access.log squid !allowedsites !prioritysites !AnonymousAccess

### squid Debian defaults
http_port 3128
hierarchy_stoplist cgi-bin ?
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320

Create the blocked and allowed sites files and some blocked and allowed sites to them.
touch /etc/squid3/allowedsites.txt
touch /etc/squid3/blockedsites.txt
touch /etc/squid3/exceptedsites.txt
touch /etc/squid3/prioritysites.txt

Some examples of entries (in the blockedsites) might be:
### How to add domains to this file
#   1.  Use only the domain name EXCLUDING the protocol prefix, i.e.
#       don't put "http://" at the start.
#   2.  Do not append a directory to the domain name, i.e. don't put
#       /index.php/path/blah.html at the end of the name.
#   3.  Prefix each entry with a single dot ".", this ensures a match of
#       example.com and www.example.com.
#   4.  If you need to match different top level domains like .com,
#       .net, .com.au for sites that have multiple top level domains to
#       the same website then add a seperate entry for each e.g.
#           .example.com
#           .example.com.au
 Note: The prioritysites list would be websites such as Antivirus updates, Windows updates etc. These sites are completely open and unrestricted for all computers so the list should be managed carefully.

Then restart squid and check for any errors.
/etc/init.d/squid3 restart

Take a look at the logs /var/log/squid3/access.log and
/var/log/squid3/cache.log as well to check squid is happy

DISCLAIMER: The information provided on this website comes without warranty of any kind and is distributed AS IS. Every effort has been made to provide the information as accurate as possible, but no warranty or fitness is implied. The information may be incomplete, may contain errors or may have become out of date. The use of this information described herein is your responsibility, and to use it in your own environments do so at your own risk.

Copyright © 2012 LINUXHOWTO.IN


  1. I salute to the admin of this diary. i actually like and that i can sure as shooting advocate this diary to my friends.
    Mp3Raid UK proxy

  2. What a nice blog...I am really very impressed to read this..Thanks to admin for posting this nice blog....WOW!!!!!
    access FileCrop in UK

  3. Bonjour
    si j ajoute un utilisateur dans ad il quicn existe dans aucun groupe et je teste mon objectif est que internet ne fonctionne pas et lorsque je l ajoute au groupe il aure des sites blacklistes and white listes et avec external acl squid si je teste elle n affiche pas les utilisateurs et leurs groupe meme avec autorisations donné heeeeeelp pleaaase