Changing Zeek’s Log Rotation Time

Log Rotation

At the end of each hour, Zeek will take the currently open log files in /opt/zeek/logs/current/ and move them over to today’s directory, compressing them with gzip in the background.

This leaves /opt/zeek/logs/ with one directory for each day of collected logs. Each holds one log file per hour. Here’s a trimmed example:

ls -Al /opt/zeek/logs/2022-08-09/

-rw-r--r-- 1 root root   267 Aug  9 01:00 capture_loss.00:00:00-01:00:00.log.gz
-rw-r--r-- 1 root root   266 Aug  9 02:00 capture_loss.01:00:00-02:00:00.log.gz
-rw-r--r-- 1 root root   265 Aug  9 03:00 capture_loss.02:00:00-03:00:00.log.gz
...
-rw-r--r-- 1 root root   265 Aug  9 23:00 capture_loss.22:00:00-23:00:00.log.gz
-rw-r--r-- 1 root root   268 Aug 10 00:00 capture_loss.23:00:00-00:00:00.log.gz
-rw-r--r-- 1 root root   835 Aug  9 01:00 conn-summary.00:00:00-01:00:00.log.gz
-rw-r--r-- 1 root root   595 Aug  9 02:00 conn-summary.01:00:00-02:00:00.log.gz
-rw-r--r-- 1 root root   798 Aug  9 03:00 conn-summary.02:00:00-03:00:00.log.gz
...
-rw-r--r-- 1 root root   786 Aug  9 22:00 conn-summary.21:00:00-22:00:00.log.gz
-rw-r--r-- 1 root root   644 Aug  9 23:00 conn-summary.22:00:00-23:00:00.log.gz
-rw-r--r-- 1 root root   787 Aug 10 00:00 conn-summary.23:00:00-00:00:00.log.gz
-rw-r--r-- 1 root root 17277 Aug  9 01:00 conn.00:00:00-01:00:00.log.gz
-rw-r--r-- 1 root root 17410 Aug  9 02:00 conn.01:00:00-02:00:00.log.gz
-rw-r--r-- 1 root root 17623 Aug  9 03:00 conn.02:00:00-03:00:00.log.gz
...

Since each log file type could have up to 24 logs in a day, that could be 250-400 files in each day’s directory.

 

Why Would We Change This?

Rotating the logs once an hour usually strikes a nice balance: each log file is not too big, and not too many really small ones. Here are some reasons why you might want to change this:

• Your analysis process needs complete log files, and waiting an hour for the next one takes too long. In this case, lower the rotation interval.
• You end up with too many small files to process. Here you can increase the rotation time. For one log a day, for example, raise the interval to 86400 seconds.
• You just want a single log file of each type instead of rotating the logs at all. In that case, set the rotation interval to 0.

 

Approach

Change Log::default_rotation_interval to the time between log rotations. While some of these will accept units like “hours” or “days”, it probably makes sense to just use seconds to be sure it will be accepted.

 

Zeek (installed with docker-zeek)

To see the current setting, run:

sudo docker exec zeek zeekctl config | grep -i logrotationinterval

To change it, edit /etc/bro/broctl.cfg (substitute your preferred editor for nano):

sudo nano /opt/zeek/etc/zeekctl.cfg

Find the following block and change the value to the number of seconds between rotations:

# Rotation interval in seconds for log files on manager (or standalone) node.
# A value of 0 disables log rotation.
LogRotationInterval = 3600

Save and exit. To confirm that the configuration file is ready to go, run:

sudo docker exec zeek zeekctl check

If all is good, put those settings in place with:

zeek reload

 

Bro

To check the current setting, run:

broctl config | grep -i logrotationinterval

, which will show the interval in seconds.

To change it, edit /etc/bro/broctl.cfg (substitute your preferred editor):

sudo nano /etc/bro/broctl.cfg

Find the following block and change the value to the number of seconds between rotations:

# Rotation interval in seconds for log files on manager (or standalone) node.
# A value of 0 disables log rotation.
LogRotationInterval = 3600

Save and exit, then run:

sudo broctl deploy

 

Zeek or Bro Installed Other Ways

If you don’t see “broctl.cfg” in /etc/bro/ or zeekctl.cfg in /opt/zeek/etc/ , search through the drive for one of those files:

sudo find / \( -name broctl.cfg -or -name zeekctl.cfg \) 2>/dev/null

In the file you locate, change the

LogRotationInterval = 3600

to the number of seconds you want, and run:

zeekctl deploy || broctl deploy

 

From the Command Line

If you’re running zeek in the foreground to process a pcap file, add the needed interval on the command line, like

zeek -r pcap_to_log.pcap local "Log::default_rotation_interval = 1 day"

 

Reference

https://docs.zeek.org/en/master/frameworks/logging.html