Beacon Analysis – The Key to Cyber Threat Hunting

Beacon analysis is by far the most effective method of threat hunting your network. In fact, I would argue that if you are not checking your network for beacon activity, you have a huge gap in your defenses that attackers will happily leverage. In this two-part series, I’ll describe what is involved with performing a beacon analysis, why it is so important in catching the bad guys, and show you some open source and commercial tools you can use to simplify the process.

 

Understanding Phone Home or Call Home Malware

Malware infected desktops, servers, and hardware can leverage a wide range of techniques to go undetected on the system. This is what makes host-based threat hunting so problematic. Unless you know for sure the system is compromised, it is easy to miss any minor telltale clues. However, the one thing all malware strains have in common is that they need to be able to communicate with their author in order to execute their marching orders.

In the past, malware authors used to connect directly to the systems that they compromised. Today, most systems sit behind a firewall, limiting the ability to access them from the Internet. So if an attacker can fool one of your employees into infecting their own system, the attacker can’t count on having direct access to the system because a firewall will most likely block their access. This is the good news.

The bad news is, attackers have found a way around this problem. The solution is to use an intermediary server called a “Command and Control” (C&C) server. The basic setup is shown in the first figure.

When a system becomes infected, it generates an outbound connection across the internet to the attacker’s C&C server. Typically this connection will try and mimic normal traffic patterns by using HTTP, HTTPS or DNS. From a cursory view, the traffic will look like normal network activity. The intent of the connection is to inform the C&C server that a new compromised system has been activated and that the system is ready and waiting for marching orders. The process will then sleep for some period of time before repeating the check in process.

When the attacker wishes to activate the compromised system, they simply cue up a command on the C&C server. The next time the compromised system checks in, they have relayed the commands and execute on whatever marching orders have been given to them. These marching orders can be anything from stealing information off of the local system (data exfiltration) to attacking some identified host out on the Internet (DDoS attack).

 

Beacon Characteristics

Within the security industry, this behavior of calling home at regular intervals is referred to as “beaconing”. While on the surface beaconing can appear similar to normal network traffic, there are some unique traits we can look for as part of a network threat hunt. These traits revolve around the timing of the communications and the packet size being used.

 

Beacon Timing

As shown in the above example, a beaconing system calls home at regular intervals. This could be as quick as every 8-10 seconds or as long as a few times a day. It really depends on how patient the attacker is and how long they feel they can avoid detection. If the attack is concerned that their malware may be detected quickly, they may beacon more frequently in order to maximize system use prior to detection. There really is no specific time interval that all attackers use, which again contributes to the difficulty in detecting beacons.

Most network activity is random in its timing. For example, you may frequently use Google to perform searches, but it is unlikely that you use it exactly at the top of the hour, every hour. You leverage Google when you need it, not at some fixed time interval. So the predictable nature of beacon timing is one of the unique characteristics we can clue in on.  

 

Beacon Packet Size

As noted in the first figure, the compromised system will spend a lot of time checking in only to find there are no marching orders for it to execute. This communication exchange of checking in and being told there is nothing to do uses a fixed set of commands. The result is that all of these sessions where the malware has nothing to do will result in identical amounts of data being exchanged. Even if the attacker takes steps to obfuscate the data, the size will remain consistent.

Most network activity is random in the amount of data exchanged in each session. For example, visiting multiple web pages on the same site will return images, text and code of various lengths. This will cause each session generated to transfer different amounts of data. So another predictable characteristic of beaconing behavior is consistency if the amount of data transferred per session.  

 

Beacon False Positives

Beaconing is a communication characteristic. It’is not good or evil, but just a way of describing the communication flow. While beaconing is heavily relied on by call home software, there are in fact times that legitimate software can exhibit beaconing behavior as well.

The most common false positive you will see is Network Time Protocol (NTP). NTP is used to ensure that the time on the local system remains accurate. NTP will beacon at a consistent interval in order to check the current time and ensure that the local system clock has not drifted. The beacon interval varies with different operating systems, but it is usually once every 15 to 60 minutes. Further, because NTP asks the same question each time (What’s the current time?), and gets back a fixed length answer, the amount of data transferred in each session is the same. So while NTP is the perfect example of beaconing behavior, it’s a vital tool that’s needed by every system and hardware device.

Luckily it is pretty easy to eliminate false positives. Our environment should be using a set of documented NTP servers to sync time. We can simply create an exception or “whitelist” these systems, thus ignoring any detected beacon activity. For example, let’s say our systems sync time with stratum2.example.com. We now know that we can safely ignore all UDP/123 traffic going to that system as part of our beacon analysis.

 

Beacon Analysis

I’m not going to lie to you. Manually performing a beacon analysis is very difficult. There are a number of challenges that need to be overcome just to get the data into a format where a proper threat hunt is possible. Here are the basic steps:

• Capture all traffic flowing through a choke point to the Internet. The best choice is usually the internal interface of your firewall.
• Capture and store enough traffic to record multiple instances of beacon activity. At a minimum, this is 12 hours of traffic. 24 hours is more ideal.
• Whitelist out any traffic that may contain beacons that you know are safe. For example, any UDP/123 traffic going to known NTP servers.
• Segregate the traffic into IP address pair combinations. For example, all traffic between 192.168.1.100 and 1.2.3.4 should go in one file, while all traffic between 192.168.1.100 and 1.2.3.5 should go in another. Note that we are not breaking out unique sessions. Each file needs to contain all traffic that was exchanged between the two identified IP addresses over the full time interval of the capture.
• Identify the time delta between the first packet in each session established between these two systems. Organize these by transport and target port that was used.
• Identify the amount of data transferred in each session. Organize these by transport and target port that was used.
• Review the data for any obvious patterns.

Clearly just getting the data into a format where it can be used for threat hunting is pretty challenging. You need to tap the network, deploy hardware capable of writing packets at wire speed and have sufficient storage for two copies of all packets (one is the original capture, the other is the one manipulated for analysis). Multiply that by the number of days you want to store the info in case you need to go back through a threat hunt (if I find something evil today I may want to go back through old data to verify when it first arrived). You then need to do multiple passes through the data to sort and organize it in a way that helps highlight beaconing behavior. Whew!

The payoff is profound visibility into your network. Have a look at the second figure where I have used tshark to extract the interesting portions of a communication session between two IP addresses.

The columns from left to right are source IP, destination IP, protocol (17 is UDP), port number (53 is the well-known port for DNS), the size of the packet, and finally the time gap between the packets shown on the screen. Two things should immediately jump out at you. First, every packet is the same size (89 bytes). This is an obvious predictable pattern. Second, the sessions are being consistently initiated about once per second, and are tightly grouped together around that one-second mark. This makes the time interval extremely predictable as well. Finally, since this is UDP/53 traffic, we expect it to be DNS. DNS has no inherent beaconing behavior like NTP, which makes this pattern extremely abnormal.

From a threat hunter’s perspective, this pattern tells me that the source IP is most likely compromised and a host-based forensic analysis is warranted. Further, I will want to ensure that my threat hunt looks careful for this behavior from any of my other systems. It’s worth noting that this is dnscat2 traffic. The tool creates an encrypted communication channel over DNS. All communications are DNS compliant, so it will happily tunnel through a DNS proxy server. Since all communications are encrypted, there is no pattern to try and find with your IDS or IPS. Shy of blind luck, performing a beacon analysis is one of your few options available for identifying when this tool is being used on your network without generating a huge number of false positives.

 

Simplifying The Beacon Analysis Process

As you can see, manually performing a beacon analysis can be a huge chore. In part two I’ll talk about RITA, an open-source tool you can leverage to dramatically simplify the process. I’ll also talk about AI-Hunter, a commercial solution, which can simplify the process even further.