Changing Zeek’s Log Rotation Time
Log Rotation
At the end of each hour, Zeek will take the currently open log files in /opt/zeek/logs/current/ and move them over to today’s directory, compressing them with gzip in the background.
This leaves /opt/zeek/logs/ with one directory for each day of collected logs. Each holds one log file per hour. Here’s a trimmed example:
ls -Al /opt/zeek/logs/2022-08-09/
-rw-r--r-- 1 root root 267 Aug 9 01:00 capture_loss.00:00:00-01:00:00.log.gz -rw-r--r-- 1 root root 266 Aug 9 02:00 capture_loss.01:00:00-02:00:00.log.gz -rw-r--r-- 1 root root 265 Aug 9 03:00 capture_loss.02:00:00-03:00:00.log.gz ... -rw-r--r-- 1 root root 265 Aug 9 23:00 capture_loss.22:00:00-23:00:00.log.gz -rw-r--r-- 1 root root 268 Aug 10 00:00 capture_loss.23:00:00-00:00:00.log.gz -rw-r--r-- 1 root root 835 Aug 9 01:00 conn-summary.00:00:00-01:00:00.log.gz -rw-r--r-- 1 root root 595 Aug 9 02:00 conn-summary.01:00:00-02:00:00.log.gz -rw-r--r-- 1 root root 798 Aug 9 03:00 conn-summary.02:00:00-03:00:00.log.gz ... -rw-r--r-- 1 root root 786 Aug 9 22:00 conn-summary.21:00:00-22:00:00.log.gz -rw-r--r-- 1 root root 644 Aug 9 23:00 conn-summary.22:00:00-23:00:00.log.gz -rw-r--r-- 1 root root 787 Aug 10 00:00 conn-summary.23:00:00-00:00:00.log.gz -rw-r--r-- 1 root root 17277 Aug 9 01:00 conn.00:00:00-01:00:00.log.gz -rw-r--r-- 1 root root 17410 Aug 9 02:00 conn.01:00:00-02:00:00.log.gz -rw-r--r-- 1 root root 17623 Aug 9 03:00 conn.02:00:00-03:00:00.log.gz ...
Since each log file type could have up to 24 logs in a day, that could be 250-400 files in each day’s directory.
Why Would We Change This?
Rotating the logs once an hour usually strikes a nice balance: each log file is not too big, and not too many really small ones. Here are some reasons why you might want to change this:
- Your analysis process needs complete log files, and waiting an hour for the next one takes too long. In this case, lower the rotation interval.
- You end up with too many small files to process. Here you can increase the rotation time. For one log a day, for example, raise the interval to 86400 seconds.
- You just want a single log file of each type instead of rotating the logs at all. In that case, set the rotation interval to 0.
Approach
Change Log::default_rotation_interval to the time between log rotations. While some of these will accept units like “hours” or “days”, it probably makes sense to just use seconds to be sure it will be accepted.
Zeek (installed with docker-zeek)
To see the current setting, run:
sudo docker exec zeek zeekctl config | grep -i logrotationinterval
To change it, edit /etc/bro/broctl.cfg (substitute your preferred editor for nano):
sudo nano /opt/zeek/etc/zeekctl.cfg
Find the following block and change the value to the number of seconds between rotations:
# Rotation interval in seconds for log files on manager (or standalone) node. # A value of 0 disables log rotation. LogRotationInterval = 3600
Save and exit. To confirm that the configuration file is ready to go, run:
sudo docker exec zeek zeekctl check
If all is good, put those settings in place with:
zeek reload
Bro
To check the current setting, run:
broctl config | grep -i logrotationinterval
, which will show the interval in seconds.
To change it, edit /etc/bro/broctl.cfg (substitute your preferred editor):
sudo nano /etc/bro/broctl.cfg
Find the following block and change the value to the number of seconds between rotations:
# Rotation interval in seconds for log files on manager (or standalone) node. # A value of 0 disables log rotation. LogRotationInterval = 3600
Save and exit, then run:
sudo broctl deploy
Zeek or Bro Installed Other Ways
If you don’t see “broctl.cfg” in /etc/bro/ or zeekctl.cfg in /opt/zeek/etc/ , search through the drive for one of those files:
sudo find / \( -name broctl.cfg -or -name zeekctl.cfg \) 2>/dev/null
In the file you locate, change the
LogRotationInterval = 3600
to the number of seconds you want, and run:
zeekctl deploy || broctl deploy
From the Command Line
If you’re running zeek in the foreground to process a pcap file, add the needed interval on the command line, like
zeek -r pcap_to_log.pcap local "Log::default_rotation_interval = 1 day"
Reference
https://docs.zeek.org/en/master/frameworks/logging.html
Bill has authored numerous articles and tools for client use. He also serves as a content author and faculty member at the SANS Institute, teaching the Linux System Administration, Perimeter Protection, Securing Linux and Unix, and Intrusion Detection tracks. Bill’s background is in network and operating system security; he was the chief architect of one commercial and two open source firewalls and is an active contributor to multiple projects in the Linux development effort. Bill’s articles and tools can be found in online journals and at http://github.com/activecm/ and http://www.stearns.org.