ProxySG authentication performance low

Last year I had an intresting case on Bluecoat/Symantec ProxySG (now Broadcom) with performance issues. Customer uses BCAAA to do authentication for every single connection.

After reviewing the authentication logs on the BCAAA servers i recognized that 80% of the authentications are made by NTLM which is not very performant. I exported the Windows log to CSV and count the events with a small python script. Furthermore there are also limitations with NTLM and it can not be cached by the client, so every time 3 way handshake.

On peak load we hat peaks up to 1,5 seconds for one authentication. The reason for the peaks are that on Microsoft servers also limitations exsisting for NTLM authentication.

Windows: Configuring MaxConcurrentAPI for NTLM Pass-Through Authentication - TechNet Articles - United States (English) - TechNet Wiki (microsoft.com)

performance tuning for NTLM authentication - Windows Server | Microsoft Docs

The relevant documentation form Bluecoat are:

Page 1150:

When the appliance receives a request that requires authentication, it consults the IWA configuration settings you have defined to determine what type of challenge to return to the client. It will try to use the strongest authentication protocol that is configured and, if the browser cannot use that protocol or if it is not configured properly, the appliance will downgrade to the next authentication protocol. For example, if you configure the IWA realm to allow Kerberos and NTLM authentication, but the user agent/browser does not support Kerberos, the appliance will automatically downgrade to NTLM.

Page 1153:

Kerberos is the recommended authentication protocol for IWA because it is more secure than NTLM or Basic and it puts the least load on your network.

Reason

#

The final reason was that not all SPNs were correct set for the service account, so BCAAA was not able to use Kerberos and fallback to NTLM.

Fix

#

To fix this register the correct FQDN of the proxy as SPN. For example: setspn -A HTTP/bcaaaUser1.acme.com AcmeDomain\BCAAAuser

Script

#

Python script to count NTLM and Kerberos from Windows Authentication Log export

import csv
import re
 
reader = csv.DictReader(open('Auditlog_Auth.csv'))
 
xlist = []
 
for l in reader:
    if l['Task Category'] == "Logon":
        if "Kerberos" in l[None][0]:
            user = ""
            hostname = ""
            xlist.append((l['Date and Time'],"Kerberos", user, hostname))
 
        if "NtLmSsp" in l[None][0]:
 
            # user search
            user = ""
            re_username = re.search(r"WINDOWSDOMAIN\\(\S+).*", l[None][0])
            if re_username:
                user = re_username.group(0)
 
            # hostname search
            hostname = ""
            re_hostname = re.search(r"Workstation Name:\s+(\S+)", l[None][0])
            if re_hostname:
                hostname = re_hostname.group(1)
 
            xlist.append((l['Date and Time'],"NtLmSsp", user, hostname))
 
k_count = 0
n_count = 0
 
n_u_list = []
n_h_list = []
 
for l in xlist:
    xdate, xauth, xuser, xhost = l
    if xauth == "Kerberos":
        k_count += 1
    else:
        n_count +=1
 
        if xhost not in n_h_list:
            n_h_list.append(xhost)
 
        if xuser not in n_u_list:
            n_u_list.append(xuser)
 
 
with open("result.csv", "w") as f:
    f.write("date, authmethod, user, hostname\n")
    for x in xlist:
        xdate, xauth, xuser, xhost = x
        f.write(f"{xdate}, {xauth}, {xuser}, {xhost}\n")
 
with open("ntlm_users.txt", "w") as f:
    for x in n_u_list:
        f.write(f"{x}\n")
 
with open("ntlm_hosts.txt", "w") as f:
    for x in n_h_list:
        f.write(f"{x}\n")
 
print(f"Kerberos {k_count} | NTLM {n_count}")

Output

#

python log_conv.py
Kerberos 186997 | NTLM 14880
```