Threat Simulation – Client Signatures (User Agent)

  Intro

This article is number 3 of 8 in a series on testing Threat Hunting software to make sure that it’s configured correctly and working successfully. If you’ve not already read the “Threat Simulation Overview and Setup” article, start there and return here to test whether your Threat hunting platform can detect odd client signatures.

 

What Traffic Are We Trying to Find?

When a web client connects to a web server it sends a User Agent, a banner that describes the software in use, like:

“Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:73.0) Gecko/20100101 Firefox/73.0”

That client is Firefox version 73.0 running on Mac OS 10.12 on a Macintosh using code from the Mozilla and Gecko projects. We can assume that identical software packages running on identical OS’s with identical libraries will have identical User Agent strings.

Now think about a piece of malware trying to communicate with a Command and Control (C2) server; what banner string should it pick? Describing itself as “Malware 1.0 (about to erase your systems)” is a little on the nose. 🙂 That said, it certainly could; a software package can describe itself with any string it likes, and on most networks that would never be noticed. The better approach would be to use a string already used by existing software – like the Firefox string above – so if anyone took the time to check they’d see a legitimate software package and hopefully ignore it.

This means we can’t just come up with whitelists of banner strings. However, we can look at how common these banner strings are. In an office where everyone has the same OS, the same support libraries, and the same web browser, we should see all of them with the exact same banner string. A piece of malware hoping to hide on that network might very well pick the banner string from a legitimate application, but it’s very unlikely that it will match the legitimate applications on this network.

Because of this, we can look at the banner strings used on the network, and focus on the rarest of them. In that group we’re likely to find the following things:

• Actual malware checking in with their C2 servers
• Applications that haven’t been recently patched and are still using an old banner string
• Applications that have been replaced with beta versions and are using a too-new banner string
• Rare applications that may not be approved for use on this network
• Applications running on unapproved devices or operating systems

 

What Tools Are Needed

This is surprisingly simple to simulate; two of the most popular command line web clients support lying about their User Agent (mostly for the time when one wishes to appear to be a different web browser because the server in question changes its behavior based on the client identity).

In the initial setup for these checks we installed both wget and curl. We’ll give command line examples for both.

 

What to Set up on the External System

You should have a web server installed on the External system listening on port 80. To test this, run either of these on the Internal system:

wget http://Ext4 -O - | less
curl -s http://Ext4 | less

Both should return an HTML page, likely claiming that “Apache has not been properly configured”. That’s just fine – we only need a web server that’s accepting requests, and don’t care what HTML page comes back.

If you get “Connection refused” or some other error code, double-check that Apache is installed and started, and the firewall allows incoming traffic on port 80.

 

What to Set up on the Internal System

Now we can place HTTP requests to the External system. We’ll use a banner string below that was once legitimate, but for a web browser from 2004. 🙂

screen -S testsignature -R

You’ll be given back a new command prompt running inside screen (so you can disconnect from it and reconnect to it at any time). In this new prompt, run:

while sleep 237 ; do curl -s --user-agent 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040124' http://Ext4 >/dev/null ; wget -q --user-agent='Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040124' "http://Ext4" -O /dev/null ; done

Once you’ve run these, you can press “Ctrl-A”, then d to disconnect from the screen session and let those commands run in the background. You can reconnect at any point with:

screen -S testsignature -R

and disconnect again with “Ctrl-A”, then “d”.

 

What You Should See on the Threat Hunting Platform

The Threat Hunting software should see and report this User-Agent string as Rare, as there’s effectively no chance it could be used by any other legitimate package on your network.

How to Troubleshoot It If You Don’t

With tcpdump

tcpdump doesn’t specifically recognize the User-Agent field in the TCP conversation, but it can display all the text sent or received if you’re willing to look down for it. By adding “-X” to the tcpdump command, it will show the entire payload in both hex and ascii:

sudo tcpdump -i eth0 -qtnp -X '(host Ext4 or host Ext6) and port 80'

This snippet shows part of the traffic from Internal to External, specifically the HTTP request and the User-Agent header (bolded, both in hex and ascii):

You can run this same command on the Internal system, the External system, and if you have command prompt access to your Threat Hunting system, there too. If you see traffic at the Internal system and the External system but not at the Threat Hunting system, double check that it’s plugged into a mirror/copy/span port on your switch.

With Zeek

Zeek records the User-Agent when it reports on HTTP connections. To see all the User-Agents seen so far this hour, use:

cat /var/spool/bro/manager/http.log | bro-cut id.orig_h user_agent | sort | uniq | sort -V
10.0.0.41    Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:73.0) Gecko/20100101 Firefox/73.0
10.0.0.41    ddclient/3.9.0
10.0.0.215   Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040124

To inspect previous compressed http logs, use:

cd /opt/bro/logs/2020-03-14/
zcat http.11\:00\:00-12\:00\:00.log.gz | bro-cut id.orig_h user_agent | sort | uniq | sort -V
10.0.0.41     Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:73.0) Gecko/20100101 Firefox/73.0
10.0.0.41     ddclient/3.9.0
2601:31d:970a:fdb0:7052:f1a2:4d2b:b8b8       Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:73.0)
Gecko/20100101 Firefox/73.0
...

cd /opt/bro/logs/2020-03-14/
zcat http.*.log.gz | bro-cut id.orig_h user_agent | sort | uniq | sort -V
10.0.0.41     Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:73.0) Gecko/20100101 Firefox/73.0
10.0.0.41     ddclient/3.9.0
10.0.0.41     trustd (unknown version) CFNetwork/811.11 Darwin/16.7.0 (x86_64)
10.0.0.41     -
...

 

Ways to Obfuscate the Traffic

The User-Agent setting is connected to neither the server to which you’re connecting nor the URI you request from that server. Changing the server name and URI, even for each request, should have no effect on your software’s ability to identify rare User-Agents.

 

Notes

If you’re interested to see what banner string is used by default, both curl and wget can show you these:

curl -s -v http://Ext4 >/dev/null
...
> User-Agent: curl/7.67.0
...


wget -d http://Ext4 -O /dev/null
...
User-Agent: Wget/1.20.3 (darwin16.7.0)
...

Modern graphical web browsers can sometimes be convinced to change their User-Agent string as well, though it commonly requires installing a plugin. Search through the plugin list for something like “User Agent Switcher”.

If you come across a User-Agent you don’t recognize, enter it Here.  That site will parse the string for you and provide a simple summary of it.

 

Links to All Posts in “Threat Simulation” Series

1. Unexpected Protocol on Non-Standard Port
2. Long Connections
3. Client Signatures (User Agent)
4. DNS
5. Beacons
6. Threat Intel
7. Certificate Issues
8. Client Signatures (TLS Signature)