Malware of the Day – Merlin C2 Data Jitter
What is Malware of the Day?
Lab Setup
Malware: MerlinC2
MITRE Tactics: TA0011 Command and Control, T1566 Phishing, T1533.005 Mark-of-the-Web Bypass
Traffic Type: HTTP/2
Connection Type: TCP
C2 Platform: MerlinC2
Origin of Sample: Active Countermeasures Threat Hunting Lab
Host Payload Delivery Method: Executable (*.exe)
Target Host: 192.168.2.115 (Windows 10 Pro x64)
Adversary Host: 24.199.110.233 (Ubuntu)
Beacon Delay: 90s
Jitter: 3s
Data Jitter: 4096 bytes
Malware of the Day Mission
To identify and share examples of post-compromise network activity in order to better detect and respond to potential network threats. Specifically we are looking for Command and Control (C2) communication channels used by attackers to obtain intelligence, issue commands, and exfiltrate data through a compromised host or hosts.
Preface: The Threat Hunting Mindset
As threat hunters, we approach our work with the mindset of a detective, maintaining an open and unbiased perspective to ensure that all potential evidence is considered and interpreted objectively. Our investigation begins when we discover a piece of evidence that suggests something might be amiss. From this point, we employ a variety of supportive tools to either refute or strengthen our case.
The process of threat hunting involves layering evidence, gradually increasing the probability of a compromise as we gather different, independently verified pieces of information. Each new piece of evidence adds to the overall picture, allowing us to build a stronger case. Our ultimate goal is to construct a case that is sufficiently solid to inform the Incident Response team of a probable compromise.
Throughout the investigation, it is crucial to remember the golden rule of threat hunting: never cross the active-passive line. This means that we must use all the tools and techniques at our disposal in a manner that does not alert the intruder to the fact that they are being investigated. By maintaining a passive approach, we can gather the necessary evidence without compromising the investigation.
By adhering to this mindset and methodology, we can effectively investigate potential compromises, providing valuable insights and enabling our organization to respond to threats in an effective manner.
Background
Merlin C2 is an open-source Command and Control (C2) framework developed by Russel van Tuyl (Ne0nd0g). Written primarily in Golang, Merlin boasts several innovative features, such as OPAQUE key exchange, encrypted JWT message authentication, and the use of HTTP/3 (QUIC) as its transport protocol. These features were explored in detail in our previous blog post.
Another particularly noteworthy feature of Merlin is its ability to “jitter the payload,” a technique it refers to as “padding.” While many in the cybersecurity community are familiar with “beacon jitter”—where an attacker randomizes the check-in time between the agent and server to mimic natural behavior—payload jitter takes this concept a step further.
Payload jitter involves appending a random amount of data (within a specified range) to every message. For example, an attacker might set the value to 4096 bytes (Merlin’s default value), meaning each packet is padded with a random amount of junk data between 0 and 4096 bytes. This randomization disrupts the predictable patterns typically associated with C2 traffic, making it more difficult to detect.
Given that this feature is both unique and enabled by default, and considering Merlin C2’s recent involvement in several threat campaigns (see here for example), it’s essential to become familiar with it. Understanding how payload jitter works will help us better recognize and mitigate its use with tools like AC-Hunter, helping to keep our networks safe and secure.
Scenario and Setup
In today’s scenario, an adversary successfully spearphished an employee at a major organization. The attacker posed as a trusted source, sending the employee what appeared to be their recent employment agreement. The email instructed the recipient to download an attached *.iso file and extract a *.xlsm file contained within it.
By embedding the malicious file within the *.iso, the attacker bypassed the Mark-of-the-Web (MotW) security feature. As a result, the *.xlsm file was not flagged by Microsoft Defender. When the employee opened the “employment offer,” they were prompted to enable macros. Despite Microsoft’s efforts to assure us VBA macros are no longer a significant security risk, the attacker’s obfuscated code executed successfully, establishing a connection to a netcat listener and granting the attacker a reverse shell on the target system (192.168.2.115).
With access to the system, the attacker launched PowerShell and executed a script that downloaded, decrypted, and extracted a GZIP archive containing a file named ctlhost.exe. Using the reverse shell, the attacker executed the file, which connected to a Command-and-Control (C2) server (24.199.110.233). This provided the attacker with a persistent foothold in the organization’s network, enabling them to proceed with their objectives.
Results
During routine threat hunting in AC-Hunter a specific connection to 24.199.110.233 caught our eye due to a number of peculiar attributes – see Image 1 below.
Image 1. AC-Hunter beacon module results overview.
Not only did this specific connection rank at the top of the threat score list, but both the connection histogram (red) and the hourly-connection graph (green) raises significant suspicions. As shown, the histogram reveals connections clustered within the range of 90 to 94, displaying a distinct “bell curve” shape. This pattern is often indicative of pseudo-randomized connection lengths, a tactic commonly used to evade detection.
The hourly-connection graph further underscores the anomaly, showing remarkable consistency in activity hour-over-hour over the entire 24-hour capture period. While some background services might produce similar patterns, the uniformity and persistence of these connections warrant deeper investigation.
Additionally, AC-Hunter provides the capability to analyze packet size frequencies, offering another layer of insight. This histogram, distinct from connection duration frequencies, is displayed in Image 2 below and reveals another interesting anomaly.
Image 2. AC-Hunter beacon module with our histogram set to packet size in bytes.
What immediately stands out is the “uniformly distributed” pattern of the frequencies. Within the range of approximately 1554 to 3256 bytes (excluding the single outlier at 3739 bytes), there are very few gaps. This indicates that the packet sizes cover a wide and continuous range of sequential values.
This behavior is unusual as it does not align with patterns typically seen in non-malicious traffic or C2 traffic. As shown in Image 3 below, most “normal” traffic tends to cluster around a few specific values, with significant gaps across the range.
Image 3. A typical packet size histogram of non-malicious traffic tends to cluster around a few specific values.
In contrast, previous C2 traffic patterns observed on the same network (specifically Cobalt Strike) exhibited a distinct “heartbeat” pattern. This pattern is characterized by a high concentration of connections at a specific lower-end value, accompanied by a small number of connections with larger packet sizes, likely indicating data transfer – see Image 4 below.
Image 4. A typical C2 packet size histogram has a clear “heartbeat” – a single value on the lower end representing the vast majority of connections.
In contrast, the packet size histogram for the connection under investigation (Image 2) does not resemble any of these patterns. It almost appears as though the payload size was deliberately manipulated.
Given this ambiguity, we proceeded to conduct an IP reputation check using AC-Hunter. By clicking on the IP and selecting Shodan from the available external reputation databases, we gathered additional insights—see Image 5 below for details.
Image 5. Shodan entry for 24.199.110.233.
As shown, the IP address is registered to DigitalOcean LLC. While DigitalOcean is a reputable company and one of the largest providers of public cloud services, it is also a frequent target for threat actors. These adversaries often exploit such platforms due to their strong reputation and the ease with which resources can be treated as disposable.
“Network for Breadth, Endpoint for Depth”
The investigation has uncovered several suspicious and unusual properties of the connection in question. The evidence gathered from AC-Hunter likely warrants alerting the Incident Response team for further investigation. Following their strict guidance and instructions – as we’re now under their jurisdiction – we can delve deeper into the endpoint to see what we may discover.
Endpoint Investigation
With the target IP identified, we used System Informer (formerly Process Hacker) to pinpoint the process responsible for the connection. After launching System Informer with administrative privileges and navigating to the Network tab, we located the IP of interest – see Image 6 below.
Image 6. System Informer simplifies the process of correlating the target host with the specific process responsible for mediating the connection.
In this case, the file mediating the connection is ctlhost.exe. While the name itself appears harmless, a closer inspection of the process details reveals that the executable is located in C:\Windows\Temp—a common red flag for suspicious activity.
Image 7. System Informer further reveals the location of the executable image that spawned the process responsible for the outbound connection.
This is concerning, as threat actors frequently write payloads to C:\Windows\Temp. This directory is commonly trusted, grants write permissions to most users, and allows malicious files to execute without immediately raising suspicion. Additionally, files in this directory are often overlooked by users and some security tools, making it a convenient temporary hiding spot for malicious activity.
Now that we’ve identified the location of the executable on disk, we can submit it to VirusTotal. The platform analyzes the binary using a wide array of antivirus engines and compares its hash against a database of known malware signatures. Upon submission, the results are alarming: 23 out of 71 vendors flagged the file as malicious – see Image 8 below.
Image 8. Virus Total results for ctlhost.exe.
Upon closer inspection, we noticed that several antivirus engines specifically reference Merlin, a modern, open-source C2 framework that has been linked to numerous recent attacks. This suspicion is further reinforced by a simple Google search for ctlhost.exe, which reveals multiple reports discussing recent campaigns by Russian threat groups. These reports indicate that the groups used Merlin and renamed the agent to ctlhost.exe—see Image 9 below for details.
Image 9. A Google search reveals the use of Merlin agents named ctlhost.exe.
Finally, by consulting Merlin’s documentation we learn that it offers an innovative feature called “padding”, allowing an attacker to append a random amount of data to each packet, thereby varying its size. This feature would perfectly explain the unusual packet size histogram we encountered in AC-Hunter.
At this point we can confidently conclude that the system is compromised. Given this, it is now appropriate for the Response team to assume full control of the situation.
Conclusion
The discovery of Merlin C2’s payload jitter feature reveals an interesting paradox in threat actors’ attempts at stealth. While randomizing packet sizes aims to avoid detection by disrupting expected patterns, this very randomization creates its own distinctive signature. The uniform distribution of packet sizes observed in our analysis stands in stark contrast to both legitimate traffic patterns and traditional C2 frameworks, making it uniquely identifiable through AC-Hunter’s histogram analysis capabilities.
This incident demonstrates the critical importance of having tools that can visualize and analyze network traffic patterns in multiple dimensions. AC-Hunter’s ability to present both temporal connection patterns and packet size distributions proved instrumental in identifying this sophisticated C2 implementation. Without these visualization capabilities, such subtle anomalies might go unnoticed among the vast amount of normal network traffic.
The evolution of C2 frameworks like Merlin, with features such as payload jitter and QUIC protocol support, underscores a crucial point for threat hunters: we cannot rely solely on known signatures and traditional pattern matching. Threat actors continuously develop new techniques to evade detection, often inverting our expectations of what “suspicious” looks like. While most malicious traffic attempts to mimic specific legitimate patterns, Merlin’s approach of introducing deliberate randomization represents a novel approach that paradoxically makes it more detectable to properly equipped analysts.
This incident reinforces the importance of maintaining a curious and adaptable mindset in threat hunting. Rather than focusing exclusively on known indicators or patterns, we must remain vigilant for anything that appears different from both legitimate traffic and known malicious patterns. The threat landscape continues to evolve, and our detection methodologies must evolve with it.
Success in modern threat hunting requires both sophisticated tools and analytical thinking. While AC-Hunter provides the technical capabilities to identify anomalous patterns, it’s the human analyst’s ability to recognize when “different” might mean “suspicious” that ultimately leads to successful threat detection. As threat actors continue to innovate, this combination of powerful analysis tools and human insight is crucial to keep our networks safe and secure.
Capture Files
PCAPs
Because… PCAPs, or it didn’t happen. 😊
The following PCAP files are packet captures taken from the same lab environment over a 1-hour time frame and a 24-hour time frame. The files were generated using Wireshark from the target host and include normal Windows OS traffic and normal network broadcast traffic. They have not been edited. The PCAPs are safe, standard PCAP files and do not include any actual malware.
Merlin Payload Jitter 1 Hour Capture
merlin_http2_d90_j3_1hr.pcapng
File Size: 82.4 MB
SHA-256 Hash: 26449099BD5F81422B95244D9F40177AE057FB7F99628EAA38C87EEC44E6BD50
Merlin Payload Jitter 24 Hour Capture
merlin_http2_d90_j3_24hr.pcapng
File Size: 2.2 GB
SHA-256 Hash: 38189230EDA372888D119472210630737DC1EF98EB47109FD0C5C3797E706EB4
Zeek Logs
If you are an AC-Hunter user, we are providing the 24-hour Zeek logs for you to import directly into AC-Hunter. The following Zeek Logs have been taken from the same lab environment over a 24-hour time frame and include normal Windows OS traffic and normal network broadcast traffic. They have not been edited. The Zeek logs are safe, standard log files and do not include any actual malware.
Importing Zeek logs into AC-Hunter example:
ssh into your AC-Hunter or RITA server and upload all the Zeek logs contained in the zip file below (all files that have the ‘.log’ extension) into a temporary directory on the server. In this example, we are uploading the Zeek logs into /tmp/merlin-c2-data-jitter-zeek-logs/
Then run the following command:
For AC-Hunter v6.x and RITA v4.x and earlier:
rita import /tmp/merlin-c2-data-jitter-zeek-logs/*.log merlin-c2-data-jitter
For RITA v5.x+:
rita import --logs=/tmp/merlin-c2-data-jitter-zeek-logs/*.log --database=merlin-c2-data-jitter
You will now have a new database in the AC-Hunter UI/web interface or RITA CLI titled “merlin-c2-data-jitter” you can select and view.
Merlin Payload Jitter 24 Hour Zeek Logs
merlin_c2_data_jitter.zip
Size: 8.3 MB
SHA256 Checksum: 0DBFBD85A9522C9E88EB3B4DE3DC5FBE06CBC35714795FDD55ACE38D1E406A8B
Discussion
Want to talk about this or anything else concerning threat hunting? Want to share how good (or not so good) other detection tools were able to detect this sample?
You are welcome to join our Discord server titled “Threat Hunter Community” to discuss topics surrounding threat hunting. We invite you to join our server here.
A huge thanks to Keith Chew and Chris Brenton for allowing me the opportunity to contribute to this awesome initiative, as well as their guidance, helpful feedback, and for being a couple of stand-up gents.
Live long and prosper!
Faan has a profound love for the natural world, technology, design, and retro aesthetics. He is incredibly grateful to have discovered cybersecurity as a path relatively late in his life, and his main interests are threat hunting and post-exploitation custom tooling, in particular C2 frameworks and RATs.