(Nearly) All You Need to Know About Proxy Autoconfiguration, WPAD and PAC files

The principle behind the automatic configuration of proxy settings is obvious – To enable users to automatically obtain their proxy settings, without the requirement of having the manually configure their browser (or internet application) settings. There are other benefits, such as the ability to quickly update proxy information, as well as the ability to specify fail-over proxies, in the event that a primary proxy is not available, and I’ll go through those as well, but first…

PAC stands for Proxy Automic Configuration, and PAC files are the files that WPAD uses to pull down the proxy information. PAC files can be published via the WPAD protocol, or alternatively they can be manually configured in the browser by providing a path or URL to their location.

WPAD stands for Web Proxy Automatic Discovery, and is a method published by either DHCP, DNS or both in order to enable browsers to automatically detect the proxy settings required for the network that they are on.

PAC Files

I’ll begin with talking about PAC files, as that is where the proxy information is actually stored, and everything else is just the distribution method of your PAC files.

PAC files are written in the Javascript language. They primarily contain the following information…

  • The proxy server(s) to use
  • The port of the proxy server(s)
  • A list of sites or hosts that the proxy bypasses (the requests go directly out to the internet, without bypassing the proxy)

An example of a pretty basic PAC file, is this (don’t worry, I’ll break it down further)…

function FindProxyForURL(url, host)

{

if (isPlainHostName(host)) { return “DIRECT”; }

if (isInNet(host, “192.168.0.0”, “255.255.255.0”)) { return “DIRECT”; }

if (shExpMatch(host, “www.host.com”)) { return “DIRECT”; }

if (shExpMatch(url, “http://www.url.com*”)) { return “DIRECT”; }

return “PROXY proxy1:8080; PROXY proxy2:8080; DIRECT”;

}

This is a standard PAC file, that retrieves the URL or host that the user’s browser has accessed, and compares it to a list of exceptions to determine whether it will allow direct access to the internet, or whether it will pass the connection through a proxy server.

If the URL or host doesn’t match anything in the exclusion list, it will then pass the connection through the first proxy.

If the first proxy (proxy1) doesn’t respond on port 8080, it will try to pass the connection through the second proxy (proxy2) on port 8080, and if that also fails, it will then allow the connection to pass out through the internet directly, as it’s least preferred option. You do not need to allow the connection to pass through to the internet directly if the proxy servers don’t respond – In fact, if you do not specifically include it as a fallback option, the connection will instead time out and the user will not be able to establish their connection until a proxy server responds.

In case you’re having difficulty identifying what does what in my PAC file example, I’ll break it down a bit further…

function FindProxyForURL(url, host)

This line obtains the URL and host information from the browser, so that it has information to compare against it’s exception list – For example, if the user was trying to access Google, then http://www.google.com/ would be the URL and www.google.com would be the host.

if (isPlainHostName(host)) { return “DIRECT”; }

This essentially checks if this is a “single label” host, which means there are no full stops (periods). If a single label host can be resolved, it’s pretty much going to be internal (such as http://intranet for the URL or just intranet for the host. If the host fits these conditions, then the connection is not passed through the proxy – This is what the return “DIRECT” part of the line means.

if (isInNet(host, “192.168.0.0”, “255.255.255.0”)) { return “DIRECT”; }

This checks if the IP address of the host is on the internal network (assuming 192.168.0.0 – 192.168.0.255 is your internal network) and therefore also bypasses the proxy for this connection.

if shExpMatch(host, “www.host.com”)) { return “DIRECT”; }

This is a direct string comparison, and grabs the host variable specified earlier, and compares it against www.host.com – If it matches, then the connection bypasses the proxy.

if shExpMatch(url, “http://www.url.com*”)) [ return “DIRECT”; }

This is another direct string comparison, but ends in a wildcard (*) which means that if the URL begins with http://www.url.com then it bypasses the proxy.

return “PROXY proxy1:8080; PROXY proxy2:8080; DIRECT”;

This is where your preferred proxy servers are listed. The connection will try proxy1 on port 8080 first, proxy2 on port 8080 second, and then go out directly to the internet if the first two proxies gave no response.

PAC files can be called anything you want to call them, except if you are using the DNS WPAD implementation, in which case it MUST be called wpad.dat in lower case.

Publishing Your PAC File

There are essentially two ways to publish your PAC file – On a file system (network share, or local machine) or via HTTP (on a web server).

The file system method is pretty simple. Just copy the PAC file to the desired location. This could be the user’s local machine, or a file server.

If you are using the user’s local machine, then I would assume this would either be a temporary measure, or that you are deploying PAC files to each of the machines for a specific requirement. The more common file system method, would be to copy the PAC file up to a file server, and reference it via a network share.

In either of the file system based scenarios, you need to reference the PAC file by appending file://// to the beginning of the location, so a file stored in the user’s C: would become file:////C:/proxy.pac – Notice the use of forward slashes, rather than backslashes in the path. This isn’t specifically required, but makes sense.

The same is true for the PAC file hosted on a file server, in that the reference would be file:////SERVER/SHARE/proxy.pac

I have seen some cases where an extra fifth slash is required after file: particular in Firefox. You’ll need to find the right balance here, depending on what your application prefers.

If you’re using the HTTP method, you need a MIME type of application/x-ns-proxy-autoconfig for the file extension, which should either be .dat or .pac and then reference the file using the full URL, for example http://server/proxy.pac

Manually Configuring PAC Files In Your Browser

This method assumes a few things about your environment…

  • Your only have a few machines to manage, and manually configuring each one is no big deal; or
  • You have a method of managing the browser configuration, such as Group Policy AND your are confident there are no internet capable application outside of your management (or they don’t matter) AND your environment does not cater for external users roaming on your network (or it’s no big deal to configure these machines as they come on to your environment, and to de-configure them before they leave)

If you aren’t happy with these conditions, then the WPAD method (in the Deploying Automatic Configuration Using WPAD section) might be the way to go for you.

For the purpose of this article, I’m only going to go through Internet Explorer 7 (which is the same for Internet Explorer 6) and Firefox. Any other compatible applications will have a very similar configuration method.

Let’s start with Internet Explorer. Fire up IE, click on Tools -> Internet Options, go to the Connections tab and click on the LAN Settings button.

You should clear any previous proxy configuration from here, and then tick Use automatic configuration script and provide the path or URL to your PAC file in here (see the section above Publishing Your PAC File if you haven’t already). You should also ensure no other tick boxes are checked, such as the Automatically detect settings box, as this will slow things down (it will be looking for the WPAD implementation, and when it can’t find it, it will load your proxy script – There is a noticeable delay).

This is similar in Firefox. Click on Tools -> Options, click on the Advanced icon up the top, click the Network tab and then click the Settings button.

Clear any previous proxy information here, select the Automatic proxy configuration URL radio button and provide the path or URL to your PAC file in here (see the section above Publishing Your PAC File if you haven’t already).

As I mentioned before, you can manage these settings using Group Policy. For Internet Explorer, there are built-in configuration options in Group Policy, but in Firefox you need to rely on the use of third party tools, or alternatively an in-house developed option. There is a fantastic Firefox setting management add-on for Group Policy called FirefoxADM which can be downloaded from SourceForge at http://sourceforge.net/project/showfiles.php?group_id=129699

Deploying Automatic Configuration Using WPAD

As mentioned at the beginning of this article, you can publish your wpad.dat file using DNS and/or DHCP, although DHCP is probably the more preferred method because it is more flexible and is more easily distributed to your client machines than DNS is.

The DNS method requires the HTTP distribution of the wpad.dat file, and also requires that a CNAME alias record called wpad is created in the root domain in DNS and points to the web server that hosts your wpad.dat file. I’ll go in to specifics shortly.

The DHCP method is much more flexible, as it supports both the file system and HTTP based methods of wpad.dat distribution, and requires that you add an extra scope option to your DHCP server.

DNS

You need to have uploaded your wpad.dat (remember, lower casing for compatibility reasons) to an HTTP server and added the MIME type of application/x-ns-proxy-autoconfig for .dat file extensions. Also, it’s important that the file can be downloaded via the IP address, rather than the hostname (which means you CAN’T use host headers) because some applications actually resolve the host themselves, the then use the IP address to obtain the wpad.dat file from the server. Basically, if you can’t download http://<ipaddress>/wpad.dat, then you’re probably going to run in to issues.

If you can get to your wpad.dat this way, then you’re nearly there… The second requirement is that you need to manage your own DNS services internally, and you need to add a CNAME alias record called wpad which points to the hostname of your HTTP server where your wpad.dat file is stored. This CNAME record needs to exist in the domain that you have recorded in your client’s DNS suffix configuration on their NIC settings. If this doesn’t exist, you need to populate that information on the NIC settings to avoid problems. If you are on a Windows domain, this should already be configured.

From the clients, ensure you can ping browse to http://wpad/wpad.dat and download the file. If you can, then skip over the next DHCP section down to the part about browser configuration for WPAD.

DHCP

This is my preferred method, because…

  • It supports both file system and HTTP based hosting of the PAC file;
  • It supports custom ports;
  • It doesn’t require internally managed DNS;
  • It doesn’t require NIC settings modification to allow remote or misconfigured machines to resolve the WPAD DNS entry

To deploy your PAC file via DHCP, you need to add an extra scope option 252 to your DHCP scope. If you are using Windows 2003 DHCP, then you can following this article – http://www.microsoft.com/technet/isa/2004/help/SRSP1_H_Create252.mspx

If you’re running a different DHCP server, you need to ensure that it supports the addition of custom scope options. If it does, create the 252 option, and then add it to your scope populating the information with the location to your PAC file, but it’s important to add a trailing space to the location of your file, as there are some cases where the last character is truncated and therefore the PAC file is not loaded correctly.

You’ll need to renew the DHCP lease on the clients in order for them to obtain this information. Unfortunately, the only way to verify that your clients are receiving this information, is to use network capturing software, such as Microsoft’s NetMon, to monitor the DHCP lease negotiation.

Configuring the browser

The last step with the WPAD implementation, is just to ensure that the Automatically detect settings box is checked in the browser (called Auto-detect proxy settings for this network in Firefox).

You can do this by Group Policy, if that’s an option, or make the change for/advise your users.

Expanding System Disks in Virtual Server 2005 R2

If anyone’s ever under estimated the anticipated size of your system disk in Virtual Server (or any environment, really) you’ll know that it’s a little painful to resize the disk AND partition neatly.

In this blog, I’ve focussed specifically on Virtual Server 2005 R2, however the same principal applies to any environment that you may have. It covers the resizing of the physical (virtual) disk itself, and then the resizing of the system partition to occupy the extra available space that you have created.

For non-system disks, you can do this quite easily from within the Windows OS, however you obviously can’t do this from your system disk when you’re using it, in which case you can follow these steps…

  1. Download VHD Resizer from http://vmtoolkit.com/files/folders/converters/entry87.aspx and install it on your Virtual Server
  2. Download BartPE Builder from http://www.nu2.nu/pebuilder/ and install it on any machine
  3. Insert the original operating system install CD, or mount the ISO (in my case, it’s Windows 2003)
  4. Open the PE Builder, and say “No” to scanning for Windows source files (because we already know where they are)
  5. Point the source path to the root of the CD drive, set the output path to the location where you’d like the files to be copied (this directory can be removed once the ISO has been created) and set the media output to ISO and select where to save it.
  6. Hit the “Build” button, and continue through any warnings/license agreements
  7. You get quite a verbose output while the files are being copied, and the ISO is being created – You should want to make sure that there are no errors or warnings, otherwise your PE ISO may not boot
  8. Close and exit the PE Builder, and copy the created ISO up to a directory on your virtual server that has been configured as a search path (you can configure Search Paths from within the Virtual Server console, under “Server Properties” -> “Search paths”
  9. You can also delete the output directory, which in my case was C:\Program Files\PE Builder\Win2k3-PE
  10. Now it’s time to shut down the VM that has the system drive you need to resize
  11. With the VM shut down, make a copy of the VHD file you want to expand, which will be the file we will work with
  12. Open VHD Resizer, and point it to the copy of your system drive VHD
  13. Provide the desired location and name of your new VHD, select the new size and choose whether to create a dynamic or fixed disk – I called my new VHD “SYSTEM-new.vhd” because we want to leave the original “SYSTEM.vhd” as our backup
  14. Click on “resize” and wait for it to complete – It took about an hour and a bit to convert a fixed 8GB disk to a dynamic 20GB disk
  15. When it’s done, rename the original VHD file to put “-backup.vhd” at the end, such as “SYSTEM-backup.vhd” and then rename the newly resized VHD to the name of the original file – You can also delete the copy of the original VHD now (in my case “Copy of SYSTEM.vhd”)
  16. Now that the disk is been resized, we also need to expand the partition so that you can use the extra space, so configure the VM to mount the BartPE ISO, and then power the VM on (make sure your VM is configured to boot from CD before HDD)
  17. As we’re just going to resize the partition, there’s no need to boot with network support, so say “No” to any messages about network support
  18. Click GO -> System -> Storage -> DiskPart, to fire up DiskPart
  19. Perform the following commands…
    • Type “list volume” to get a list of the available volumes
    • Identify the volume you wish to extend and then type “select volume <volumenumber>”
    • Type “extend”
  20. Type “list volumes” again, to make sure it reports the new size
  21. Click GO -> Shutdown -> Shutdown
  22. Unmount the BartPE ISO
  23. Power your VM back up, and then view the disk details from within Disk Management to verify that your settings are correct – You may also be prompted to reboot as a system change may have been detected.