Posted by: rolande | May 10, 2010

Avoiding 407 Auth Prompts for Embedded Web Page Content

BlueCoat ShieldIn a corporate network environment, having the ability to authenticate and authorize users who require access to Web content you do not normally permit to the general user population is not a difficult feature to implement with BlueCoat ProxySG. However, the annoying side effect is for all of the users who do not have access and do not submit their credentials, they will constantly see authentication popups in their web browser on sites that have embedded content that is served from other URLs that are included in these blocked categories. This can be rather annoying, especially when there are multiple embedded objects and each time you click ‘Cancel’ another prompt pops up.

As you begin to scale a solution like this to more and more users across many load balanced proxies with more and more traffic, this problem becomes much worse. As the desire to provide additional “Exception” categories increases, the likelihood of unauthorized users running into popups increases.

So, the question is “How can you have your cake and eat it too?” Well, I came up with a solution using the built-in functionality of the SG Policy engine. It took a considerable amount of time to work through the SG behavior to understand what is going on under the covers to come up with a creative way to solve this problem. I’ll try to do my best to explain how this works and the various caveats and avoid just posting the solution.

This solution uses a combination of policy rules and custom Exception Messages. The end result is a kind of virtual authentication method that provides a seamless redirect for the client to the ultimate content they are trying to get to. You may argue that you can just use one of the various virtual auth methods included in the ProxySG itself, plus the credentials can be encrypted and then a disposable token is used instead of the actual credentials. The problem with the virtual auth methods is that they use a session cookie. That session cookie is specific for that user on the specific proxy they are currently forwarding to. In a high scale, globally, and locally load balanced proxy environment, the user can not be guaranteed persistence against the same proxy server for the entire duration of a session. So, the session cookie is useless as soon as the client gets flipped to another proxy at another data center.

On a side note, it was requested of BlueCoat to consider a feature to synchronize session cookies/auth across an array of proxies to provide this transparency to avoid the persistence pitfalls. One of the Product Managers understood the value of this capability but quickly shot the idea down due to specific technical issues that would be quite difficult/expensive to overcome and ultimately does not really generate much income for BlueCoat.

You could also argue the use of the BCAAA functionality and use the NTLM auth capability. There are several problems with that approach. NTLM does not work well on non-Microsoft platforms. NTLM does not work through a proxy. If I have clients being transparently proxied or I am bringing a Business Partner or new acquisition across a services type infrastructure, it is quite difficult to extend any services such as this that make use of NTLM auth. Additional maintenance for a server infrastructure for BCAAA agents is costly just for an auth solution for exceptions. BCAAA does not scale very well either.

In a large scale environment, the use of the standard 407 Proxy Auth response creates a fairly easy solution to provide seamless authentication across multiple proxies without having to annoy an end user with multiple authentication prompts. Once the auth credentials have been set the browser will send the Proxy-Authorization header with all requests to the same proxy server hostname and port. As a client flips between different categories of exception content, they can be transparently authorized without the user having to be prompted again. The downside is that the 407 challenge/response is not encrypted. A more strategic solution would be to use desktop certificates that tie into a global LDAP infrastructure so the authentication component would become completely transparent to the user and highly secured.

So, given that the Basic Auth solution currently scales the best, the goal was to find a way to stop annoying the users with auth popups for all the websites with embedded content in those exception categories.

By default, the easiest thing to do to enable authentication is to set a simple policy rule that matches on categories and forces authentication. The problem is that regardless of how that content is requested it will generate a 407 auth popup in a users browser every time. The question is how can we suppress the 407 responses when the content is embedded and the user is not yet authenticated? I would prefer to send a 403 response instead of the 407, unless the user specifically browses to a site in the exception category.

The solution I came up with uses a custom Exception Message denying access for requests to Exception websites, unless the Proxy-Authorization header is included in the request. If the user specifically browsed to a blocked Exception site, they would see the Exception message in their browser that includes a big link that says If you have Exception access: “Click Here to Authenticate”. The link is dynamically generated in the Exception message using the $(url) variable and it is appended with a custom string to designate that the client wishes to authenticate. This string is matched in the Policy to enforce authentication. By using the same URL the client tried to go to and appending the custom string, we can guarantee that the forwarding path within the PAC file is maintained and the 407 Proxy Auth header will be set properly to be used when the user is redirected back to the original website. Once the user clicks on the Authentication Link, they will now get the 407 response from the proxy and the browser will present the authentication prompt. Once the user successfully authenticates, they will be returned a second custom Exception message which indicates that they were successfully authenticated and it includes a Meta Refresh that has the $(request.header.Referer.url) variable embedded. This will seamlessly redirect the client’s browser back to the actual website they were trying to go to. At this point, the browser will make the request to the actual website and include the Proxy-Authorization header. The proxy policy will match on the existence of the Proxy Auth header and the Exception categories and will force the Authentication. Now that the Proxy Auth header has been set with the users valid credentials the proxy can transparently authenticate the user and ultimately perform the authorization for that users credentials to ensure they have the necessary exception to access the website.

Following is a break down of the policy to achieve this solution:

  1. The first layer is the Authentication Layer. We only authenticate the request if the user is already sending the ‘Proxy-Authorization’ header with requests that are going to Exception categories or if the URL request includes the custom string ‘proxyauth.domain.com’. The URL the client requests must be in a DENY state in order for the authenticate(realm) option to actually trigger. So by matching a Regex ‘proxyauth.domain.com$’ and setting it to DENY we achieve that requirement. Finally, since we use an alternate forwarding path for HTTPS traffic, we need to trigger the Auth when a client gets redirected to SSL.
  2. The second layer is our standard Web Access policy layer. The first rule matches the Regex string and checks if the user is already sending the Proxy-Authorization header. If so, then the proxy returns an Exception Message that provides a courtesy Redirect to the originally requested page.
  3. The second rule matches the Regex string and verifies that the Proxy-Authorization header is not included in the request. If so, then the Block Message is returned that includes the Virtual URL link to force the authentication.
  4. The remaining rules define our Blacklist or Exception list. Note that the rule for the Exception categories matches only if the client is not sending the Proxy-Authorization header.
<Proxy>
 url.regex="proxyauth.domain.com$" authenticate(realm) authenticate.force(yes) authenticate.mode(auto)
 request.header.Proxy-Authorization=".*" category=(Email, Gambling, "Online Storage", "Social Networking") authenticate(realm) authenticate.force(yes) authenticate.mode(auto)
 condition=HTTPS_Auth request.header.Proxy-Authorization=!".*" authenticate(sso) authenticate.force(yes) authenticate.mode(auto)

<Proxy>
 url.regex="proxyauth.domain.com$" request.header.Proxy-Authorization=".*" force_exception(user_defined.auth_redirect)
 DENY url.regex="proxyauth.domain.com$" request.header.Proxy-Authorization=!".*"
 request.header.Proxy-Authorization=!".*" category=(Email, Gambling, "Online Storage", "Social Networking") exception(user_defined.content_filter_unauthorized)
 ALLOW condition=BCWF_ALLOWed_Streaming
 DENY category=("Adult/Mature Content", "Audio/Video Clips", "Chat/Instant Messaging", Email, Extreme, Gambling, Hacking, "Illegal Drugs", Illegal/Questionable, "Internet Telephony", "Online Storage", "Pay to Surf", "Peer-to-Peer (P2P)", Personals/Dating, Phishing, Pornography, "Potentially Unwanted Software", "Proxy Avoidance", "Radio/Audio Streams", "Remote Access Tools", "Social Networking", "Spyware Effects/Privacy Concerns", "Spyware/Malware Sources", Suspicious, "TV/Video Streams", Violence/Hate/Racism, Weapons)

<Proxy BlueCoat_Exception_Authorization>
 ALLOW category=Email attribute.ProxyContextFlag01=TRUE
 ALLOW category="Social Networking" attribute.ProxyContextFlag02=TRUE
 ALLOW category="Online Storage" attribute.ProxyContextFlag03=TRUE
 ALLOW category=Gambling attribute.ProxyContextFlag04=TRUE
 ALLOW category=("Audio/Video Clips") attribute.ProxyContextFlag05=TRUE
 DENY request.header.Proxy-Authorization=".*" category=(Email, Gambling, "Online Storage", "Social Networking", "Audio/Video Clips")

When the user clicks on the Virtual Auth URL link in the first block message, the proxy will return a 407 Proxy Auth response to the user.

Once the user authenticates in the popup auth box, the proxy will then match the first rule and return the Auth Redirect exception message which says they successfully authenticated and provides a courtesy redirect back to the originally requested site.  The originally requested website is embedded into a Meta Refresh directive in the Redirect Exception message. The $(request.header.Referer.url) variable is used to populate the necessary fields in this message to provide the Redirect and the manual HREF link to the originally requested site.

If the users credentials fail to authorize with the proper attribute, after they have authenticated, when the browser redirects to the originally requested URL, then the proxy will return the default Content Filter Denied message to the user.

Contents of the Exception Messages

Content Filter Unauthorized Response Code: 403 Forbidden

<html>
<head>
<title>$(category) Content Not Authorized</title>
</head>
<body>
<font face="Helvetica">
<big><strong>$(exception.company_name)</strong></big><br>
<h3 align="center">$(quot)$(cs-categories)$(quot) content is only permitted on Exception by
Company policy.<!--<span class="hiddenSpellError" pre=""-->h3>
<br />
<center>
<big><strong>If you have Exception Access:</strong></big></font><br /><br />
<table border=2 cellpadding="5" cellspacing="5">
<tr>
<span style="color: red; font-family: Helvetica;">
<h2>&nbsp; <a href="$(url)&auth=proxyauth.domain.com">Click Here to Authenticate</a></h2>
</span>
</tr>
</table>
<br />
<p /><p /><p /><p />
<center>
<table border="0" cellpadding="5" cellspacing="5" width="80%">
<tr>
<td><font face="Verdana">You have attempted to access an Internet site that is only authorized by Exception through the Company's computers, gateways, or systems.</font>
</td>
</tr>
</table>
</center>
<br />
</body>
</html>

Auth Redirect Response Code: 200 OK

<html>
<head>
<title>Exception Access Authenticated. Redirecting to $(request.header.Referer.url)</title>
<meta http-equiv="refresh" content="0;url=$(request.header.Referer.url)" />
</head>
<body>
<font face="Helvetica">
<big><strong>$(exception.company_name)</strong></big><br>
<h3 align="center">Your Exception request to $(quot)<a href="$(request.header.Referer.url)">$(request.header.Referer.url)</a>$(quot) has been <br /><br />
<center><font color="green">Successfully Authenticated</font></center></h3>
</font>
<br />
<br />
<font face="Verdana">
<center><big>Your browser will automatically be redirected to the site you requested...</big><br /><br />
<p />
<p />
<strong><a href="$(request.header.Referer.url)">Click Here</a> if your browser does not automatically redirect you.</strong></center>
</font>
<br />
</body>
</html>
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: