(Nearly) All You Need to Know About Proxy Autoconfiguration, WPAD and PAC files

The principle behind the automatic configuration of proxy settings is obvious – To enable users to automatically obtain their proxy settings, without the requirement of having the manually configure their browser (or internet application) settings. There are other benefits, such as the ability to quickly update proxy information, as well as the ability to specify fail-over proxies, in the event that a primary proxy is not available, and I’ll go through those as well, but first…

PAC stands for Proxy Automic Configuration, and PAC files are the files that WPAD uses to pull down the proxy information. PAC files can be published via the WPAD protocol, or alternatively they can be manually configured in the browser by providing a path or URL to their location.

WPAD stands for Web Proxy Automatic Discovery, and is a method published by either DHCP, DNS or both in order to enable browsers to automatically detect the proxy settings required for the network that they are on.

PAC Files

I’ll begin with talking about PAC files, as that is where the proxy information is actually stored, and everything else is just the distribution method of your PAC files.

PAC files are written in the Javascript language. They primarily contain the following information…

  • The proxy server(s) to use
  • The port of the proxy server(s)
  • A list of sites or hosts that the proxy bypasses (the requests go directly out to the internet, without bypassing the proxy)

An example of a pretty basic PAC file, is this (don’t worry, I’ll break it down further)…

function FindProxyForURL(url, host)

{

if (isPlainHostName(host)) { return “DIRECT”; }

if (isInNet(host, “192.168.0.0”, “255.255.255.0”)) { return “DIRECT”; }

if (shExpMatch(host, “www.host.com”)) { return “DIRECT”; }

if (shExpMatch(url, “http://www.url.com*”)) { return “DIRECT”; }

return “PROXY proxy1:8080; PROXY proxy2:8080; DIRECT”;

}

This is a standard PAC file, that retrieves the URL or host that the user’s browser has accessed, and compares it to a list of exceptions to determine whether it will allow direct access to the internet, or whether it will pass the connection through a proxy server.

If the URL or host doesn’t match anything in the exclusion list, it will then pass the connection through the first proxy.

If the first proxy (proxy1) doesn’t respond on port 8080, it will try to pass the connection through the second proxy (proxy2) on port 8080, and if that also fails, it will then allow the connection to pass out through the internet directly, as it’s least preferred option. You do not need to allow the connection to pass through to the internet directly if the proxy servers don’t respond – In fact, if you do not specifically include it as a fallback option, the connection will instead time out and the user will not be able to establish their connection until a proxy server responds.

In case you’re having difficulty identifying what does what in my PAC file example, I’ll break it down a bit further…

function FindProxyForURL(url, host)

This line obtains the URL and host information from the browser, so that it has information to compare against it’s exception list – For example, if the user was trying to access Google, then http://www.google.com/ would be the URL and www.google.com would be the host.

if (isPlainHostName(host)) { return “DIRECT”; }

This essentially checks if this is a “single label” host, which means there are no full stops (periods). If a single label host can be resolved, it’s pretty much going to be internal (such as http://intranet for the URL or just intranet for the host. If the host fits these conditions, then the connection is not passed through the proxy – This is what the return “DIRECT” part of the line means.

if (isInNet(host, “192.168.0.0”, “255.255.255.0”)) { return “DIRECT”; }

This checks if the IP address of the host is on the internal network (assuming 192.168.0.0 – 192.168.0.255 is your internal network) and therefore also bypasses the proxy for this connection.

if shExpMatch(host, “www.host.com”)) { return “DIRECT”; }

This is a direct string comparison, and grabs the host variable specified earlier, and compares it against www.host.com – If it matches, then the connection bypasses the proxy.

if shExpMatch(url, “http://www.url.com*”)) [ return “DIRECT”; }

This is another direct string comparison, but ends in a wildcard (*) which means that if the URL begins with http://www.url.com then it bypasses the proxy.

return “PROXY proxy1:8080; PROXY proxy2:8080; DIRECT”;

This is where your preferred proxy servers are listed. The connection will try proxy1 on port 8080 first, proxy2 on port 8080 second, and then go out directly to the internet if the first two proxies gave no response.

PAC files can be called anything you want to call them, except if you are using the DNS WPAD implementation, in which case it MUST be called wpad.dat in lower case.

Publishing Your PAC File

There are essentially two ways to publish your PAC file – On a file system (network share, or local machine) or via HTTP (on a web server).

The file system method is pretty simple. Just copy the PAC file to the desired location. This could be the user’s local machine, or a file server.

If you are using the user’s local machine, then I would assume this would either be a temporary measure, or that you are deploying PAC files to each of the machines for a specific requirement. The more common file system method, would be to copy the PAC file up to a file server, and reference it via a network share.

In either of the file system based scenarios, you need to reference the PAC file by appending file://// to the beginning of the location, so a file stored in the user’s C: would become file:////C:/proxy.pac – Notice the use of forward slashes, rather than backslashes in the path. This isn’t specifically required, but makes sense.

The same is true for the PAC file hosted on a file server, in that the reference would be file:////SERVER/SHARE/proxy.pac

I have seen some cases where an extra fifth slash is required after file: particular in Firefox. You’ll need to find the right balance here, depending on what your application prefers.

If you’re using the HTTP method, you need a MIME type of application/x-ns-proxy-autoconfig for the file extension, which should either be .dat or .pac and then reference the file using the full URL, for example http://server/proxy.pac

Manually Configuring PAC Files In Your Browser

This method assumes a few things about your environment…

  • Your only have a few machines to manage, and manually configuring each one is no big deal; or
  • You have a method of managing the browser configuration, such as Group Policy AND your are confident there are no internet capable application outside of your management (or they don’t matter) AND your environment does not cater for external users roaming on your network (or it’s no big deal to configure these machines as they come on to your environment, and to de-configure them before they leave)

If you aren’t happy with these conditions, then the WPAD method (in the Deploying Automatic Configuration Using WPAD section) might be the way to go for you.

For the purpose of this article, I’m only going to go through Internet Explorer 7 (which is the same for Internet Explorer 6) and Firefox. Any other compatible applications will have a very similar configuration method.

Let’s start with Internet Explorer. Fire up IE, click on Tools -> Internet Options, go to the Connections tab and click on the LAN Settings button.

You should clear any previous proxy configuration from here, and then tick Use automatic configuration script and provide the path or URL to your PAC file in here (see the section above Publishing Your PAC File if you haven’t already). You should also ensure no other tick boxes are checked, such as the Automatically detect settings box, as this will slow things down (it will be looking for the WPAD implementation, and when it can’t find it, it will load your proxy script – There is a noticeable delay).

This is similar in Firefox. Click on Tools -> Options, click on the Advanced icon up the top, click the Network tab and then click the Settings button.

Clear any previous proxy information here, select the Automatic proxy configuration URL radio button and provide the path or URL to your PAC file in here (see the section above Publishing Your PAC File if you haven’t already).

As I mentioned before, you can manage these settings using Group Policy. For Internet Explorer, there are built-in configuration options in Group Policy, but in Firefox you need to rely on the use of third party tools, or alternatively an in-house developed option. There is a fantastic Firefox setting management add-on for Group Policy called FirefoxADM which can be downloaded from SourceForge at http://sourceforge.net/project/showfiles.php?group_id=129699

Deploying Automatic Configuration Using WPAD

As mentioned at the beginning of this article, you can publish your wpad.dat file using DNS and/or DHCP, although DHCP is probably the more preferred method because it is more flexible and is more easily distributed to your client machines than DNS is.

The DNS method requires the HTTP distribution of the wpad.dat file, and also requires that a CNAME alias record called wpad is created in the root domain in DNS and points to the web server that hosts your wpad.dat file. I’ll go in to specifics shortly.

The DHCP method is much more flexible, as it supports both the file system and HTTP based methods of wpad.dat distribution, and requires that you add an extra scope option to your DHCP server.

DNS

You need to have uploaded your wpad.dat (remember, lower casing for compatibility reasons) to an HTTP server and added the MIME type of application/x-ns-proxy-autoconfig for .dat file extensions. Also, it’s important that the file can be downloaded via the IP address, rather than the hostname (which means you CAN’T use host headers) because some applications actually resolve the host themselves, the then use the IP address to obtain the wpad.dat file from the server. Basically, if you can’t download http://<ipaddress>/wpad.dat, then you’re probably going to run in to issues.

If you can get to your wpad.dat this way, then you’re nearly there… The second requirement is that you need to manage your own DNS services internally, and you need to add a CNAME alias record called wpad which points to the hostname of your HTTP server where your wpad.dat file is stored. This CNAME record needs to exist in the domain that you have recorded in your client’s DNS suffix configuration on their NIC settings. If this doesn’t exist, you need to populate that information on the NIC settings to avoid problems. If you are on a Windows domain, this should already be configured.

From the clients, ensure you can ping browse to http://wpad/wpad.dat and download the file. If you can, then skip over the next DHCP section down to the part about browser configuration for WPAD.

DHCP

This is my preferred method, because…

  • It supports both file system and HTTP based hosting of the PAC file;
  • It supports custom ports;
  • It doesn’t require internally managed DNS;
  • It doesn’t require NIC settings modification to allow remote or misconfigured machines to resolve the WPAD DNS entry

To deploy your PAC file via DHCP, you need to add an extra scope option 252 to your DHCP scope. If you are using Windows 2003 DHCP, then you can following this article – http://www.microsoft.com/technet/isa/2004/help/SRSP1_H_Create252.mspx

If you’re running a different DHCP server, you need to ensure that it supports the addition of custom scope options. If it does, create the 252 option, and then add it to your scope populating the information with the location to your PAC file, but it’s important to add a trailing space to the location of your file, as there are some cases where the last character is truncated and therefore the PAC file is not loaded correctly.

You’ll need to renew the DHCP lease on the clients in order for them to obtain this information. Unfortunately, the only way to verify that your clients are receiving this information, is to use network capturing software, such as Microsoft’s NetMon, to monitor the DHCP lease negotiation.

Configuring the browser

The last step with the WPAD implementation, is just to ensure that the Automatically detect settings box is checked in the browser (called Auto-detect proxy settings for this network in Firefox).

You can do this by Group Policy, if that’s an option, or make the change for/advise your users.

48 thoughts on “(Nearly) All You Need to Know About Proxy Autoconfiguration, WPAD and PAC files”

  3. Pingback: Configurazione automatica proxy | Problemi e Soluzioni di un sistemista informatico

  4. Did not know if you were still monitoring this or not, but in case you are here is my question: We are using the proxy pac file stored on local machines outside of our network as a ‘whitelist’, anything that does not match one of the listed sites gets redirected back to 127.0.0.1. The problem I am having is when I update the proxy file for a new web address, the update does not take affect, or it is not recognized by IE. Is there something I should be doing to re-register it locally?

    Reply

  6. Hi – I am using Firefox v3 along with a wpad.dat on a web/IIS server. I have the DNS records setup to point to my IIS server. From the XP client I get prompted to download the wpad.dat file when I type http:///wpad.dat in the address bar, so I think everything is configured properly. My question is why doesn’t the Firefox auto-detect work but Internet Explorer works fine?

    htttp://wpad/wpad.dat also prompts as a file download in Firefox but when auto-detect is selected and I restart FF it can’t get anywhere, any suggestions? Thanks.

    Reply

    1. Hi Jim,

      Try manually setting Firefox to use http://wpad/wpad.dat and see if this works. If it doesn’t work using this method, then the problem lies with the wpad.dat file itself rather than the delivery method.

      Reply

  7. How can I know if me or a customer has been hijacked?

    I mean, apart from “configuration” of WPAC in IE where can I see current “status” of WPAC in IE?

    Reply

    1. Hi SuD,

      As far as I know, there’s no feature in any browser that will show you that option easily.

      What you can do though, is run netstat -b from the command line of the machine, and check the connections that your browser is making. There are a few GUI tools you can use that might make things a bit easier – I recommend TCPView

      Reply

  8. Excellent stuff this..

    One question though.

    I am considdering changing the wpad.dat file our ISA server issue, however this seem to be somewhat complex in comparison to your example above. My main goal would be to implement failover via inserting a second proxy in the proxy array (we have 3 isa servers worldwide).

    I don’t know if you are familiar with the ISA servers wpad.dat (its a coupple of hundred lines long), but in case you are is there any catches to exchanging it with a shorter and simpler one (like a modified version of yours).

    Reply

    1. Hi Michael,

      I am somewhat familiar with the wpad.dat file in ISA, although I do exactly what you are suggesting, in creating my own WPAD file instead.

      There aren’t any fundamental issues with this. You’ll obviously no longer make changes via the ISA GUI for things that it will normally put in the wpad.dat file itself, but I prefer to have direct control over it myself anyway; it means it’s easier to troubleshoot!

      Reply
  9. Pingback: Setting up proxy autodetect « Michael Ellerbeck

  10. We are using wpad.dat via DHCP (scope id 252) and found when a client is static IP configured, IE automatically detect settings doesn’t work.

    Reply

  11. Hi Guruprasad,

    This is, of course, the expected behaviour in that if you are not using DHCP to assign client information, they cannot obtain the WPAD options defined in your 252 scope option either.

    The recommendation is to either use DHCP reservations in place of static IP address, or to use DNS instead of DHCP to publish your proxy settings to your clients (or even a combination of both).

    Reply

  12. This is a great post! I hope it is still being monitored as I have a question.

    If I wanted a specific network to go through a different proxy, how could I add this to my existing wpad.dat?

    Thanks,
    Hamid

    Reply

  13. Hi Hamid,

    You can use this method, which will resolve the local IP address of your WPAD client, and then return a proxy based on the subnet that the client is in: –

    clientip=dnsResolve(host)

    if (isInNet(clientip, “10.0.0.0”, “255.255.255.0”)) { return “PROXY proxy1:8080” }

    Reply

  14. Hi, and happy NY,

    We use a HTTP URL based proxy.pac in IE6 browsers. My question, if you are at all able to answer, is how often or how does the browser query that particular network file. Is it cached on teh machine locally when the browser is opened or is it queried across the LAN / WAN for every url request?

    many thanks in advance.

    Danny

    Reply

  15. Hi Danny,

    The proxy.pac file is not queried per URL request; it is queried based on your browser session. In Internet Explorer, the PAC is cached for a period of time after the browser loads the .pac file.

    There are registry settings to modify the cache period for IE.

    Reply

  16. Very userfull post,

    Still one question. When de do the setup using DHCP and the ‘252’ option, this means that every PC has to be verified if the option ‘auto-detect proxu-settings’ is checked or not.

    I’m looking for a system where I can push our proxy-settings to every PC that enters our network and receive a DHCP address.

    Thanks in advance !

    Reply

  17. Hi Kristof,

    You have at least two options in this scenario: –

    1) If all of the machines under your management are on an Active Directory domain, you can use group policy to enforce the Internet Explorer auto-detect checkbox

    2) You can investigate the use of a transparent proxy, which effectively intercepts all outbound internet traffic on port 80 and redirects it automatically through your proxy server – This requires no client-side configuration, but there are other disadvantages that you should research

    You can also use a combination of the two, or develop your own solution to the problem using scripting or simply advising external users that might not be in your Active Directory domain to tick the auto-detect box manually.

    I hope this helps.

    Reply

  18. To go along with your reply to Guruprasad, is it possible to run a combination of the DHCP option 252 configuration as well as DNS (CNAME record for wpad)? If so, are there any advantages or disadvantages to this?
    We’ve been using DNS to resolve wpad for clients, but I’m looking to change this over to DHCP because it seems like it would be a more reliable option. Any recommendations or tips would be greatly appreciated!
    Great article BTW!

    Reply

  19. Hi Chad,

    You can use a combination of DHCP and DNS in your environment if you like, however be aware that different browser and browser versions will prefer one option over the other.

    Using DHCP you can assign a different WPAD file to clients in different DHCP scopes to redirect them to different proxy servers. With DNS, to accomplish this same result, you’d need to build the client location logic in to the WPAD file itself (see my response to Hamid earlier) as you can only provide one WPAD file per domain.

    Reply

  20. Thanks for the reply! Would you mind elaborating more on the different browsers and browser versions preference or link to further reading on this? I’ve searched online but haven’t found any details as of yet.

    Reply

  21. If you haven’t already, check out the Wikipedia article for WPAD, as it’s quite good – http://en.wikipedia.org/wiki/Web_Proxy_Autodiscovery_Protocol

    Regarding the browser implementations of WPAD, my understanding is that the WPAD draft states that the browser should first try DHCP before trying DNS.

    I can’t remember if I read somewhere, or discovered through personal use, that a version of Firefox I was running preferred DNS to DHCP (i.e. would check DNS before checking DHCP).

    I’d suggest checking the release notes and technical documentation for each browser you intend to deploy to verify how it handles WPAD requests.

    Reply

  24. Hi Mat

    Very useful article, thanks!

    Question: How do you deal with laptops in the internet, not finding the proxy.pac file on the internal webserver becouse there are not allowed to? There are extremly long timeouts depending on the webbrowser. I considered providing an nearly empty pac file (only “return “DIRECT”) on an external webserver which is resolved with anonther ip address on the external DNS Servers then on the internal dns servers.

    Regards
    Stefan

    Reply

  25. Hi Bruce,

    You can certainly configure exceptions in the WPAD file. In fact, the article above even has some examples. 🙂

    Reply

  26. Hi Stefan,

    The timeout shouldn’t be too noticeable, as it only occurs when the browser is first opened. Once the WPAD file cannot be located, the rest of the browsing experience isn’t affected by the WPAD file not being accessible.

    You can certainly point the browser to a “blank” WPAD file using split DNS if you wish, however typically this isn’t required.

    Generally, the convenience of not having to manually configure the proxies in a large, distributed and mobile organisation outweighs the initial delay while the browser tries to locate the WPAD file.

    Reply

  27. Hi Mat

    I have got my web server configured with https: can I still use this option or do I need to configure an http webserver ?

    Reply

  28. Is there a way for me to configure the PAC file and/or configure the browser to failover to the next proxy server should the first proxy server does not respond in a certain time (say 15 secs).

    Thank you for your help with this.

    Reply

  30. My client is not pointing the exact ISA firewall which i mentioned in the DHCP server, instead pointing to another firewall. Can you please help me how to solve this issue. (prabu.babu@tatacoffee.com).

    Reply

  31. don’t no if this is still being checked or not, but is there anywhere I can check on the local PC to see what version of the wpad file it has? For example if I make a change to wpad, i would like to somehow check something on the PC to see if it was the new wpad or see if it has a cached version of the old one?

    Reply

  34. Chuttu Shah, I am not sure if the timeout values are configurable. My understanding is that the browser makes this decision.

    Reply

  36. Hi Mat, wonder if you could help me please?

    I have a Win2008 server running DHCP with WPAD (252) enabled on the scope pointing to our proxy server. All PC clients are picking up the proxy settting automatically and can browse onto the web no problem without manual intervention.

    An additional requirement surfaced asking to perform the same functionality, but for Ipads and Iphones. At present (using the same DHCP scopes) the Ipads and Iphones will not pick up the proxy address automatically (manually works fine) What do I need to do to get Ipads/Iphones proxy setting automatically, but without disrupting the PC side of things? Many Thanks

    Reply

  37. Hi Zulu,

    Maybe something in this thread will help you (it’s for Mac OS X, but I reckon the protocol is still being handled the same way): –

    http://forums.macrumors.com/showthread.php?t=991781

    The options provided were to manually append a null character (“%00”) to either the wpad.dat file, or as a redirect in IIS, or as the second option was to use the DNS alias WPAD

    Reply

  38. I have tried using your guide and have can’t get the proxy to work. I need it more for my laptop users so that when they are away from the network it finds the internet automatically and then when they come back to work and connect it automatically will work. Right now users have to manually change the “auto detect” on/off. Also our proxy server requests l/p every time IE is opened.

    Reply

  39. Is there any way to reduce the time delay taken by IE to wait for the proxy server to respond with an url.

    Reply

  40. Dear sir,

    We are using DHCP Wpad in our infra, so is it possible to configure bypass proxy server for local server?

    Actually all hits are coming on ISA, i want to set bypass setting for local server. Is it possible, if yes pls suggest.

    Reply

  41. I have two ISA servers with different ISP. I need to establish isa failover for my clients. I need some help with the pac file. What I want to achieve is this:

    for all internal addresses bypass proxy
    use proxy 1 external URL
    use proxy 2 for external when proxy 1 is unavailable

    Thanks

    Reply

  42. Hi

    Could someone please calrify how domain joined clients are handled, when they are outside the domain (field operators etc) when using WPAD? HEre the clients will optain a DNS outside the domain?

    Please do not hesitate to contact me

    br

    Lasse

    Reply

  44. Hi !,

    Quick question, I already have a proxy pac in place which handles traffic for a particular destination e.g (Site A), I have been requested to write a new pac which directs traffic out over (Site B) but if traffic is for (Site A) then calls the existing proxy pac. In effect calling a proxy pac from within a proxy pac, is this possible. Thanks

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *