[Maarten Van Horenbeeck] [Information Security] [Resources]

Configuring auditd on Debian (process execution logging)

In order to effectively detect, and respond to, security incidents, it's important to have proper logging data for security relevant events that take place on a system. These types of logs may include process execution, file access, user creation, and others.

In particular process execution logs are incredibly valuable. When acquired from systems, they can help create a trace of all actions that took place on the system. This can then help determine timelines, and determine appropriate next steps for investigation.

The audit system at a high level

A key mechanism to get this type of logging granularity on a Linux system is the Audit subsystem, part of the Linux kernel. The Audit subsystem can be used to log kernel events, such as specific system calls, and user events. It consists of a kernel subsystem which implements the logging, and an audit daemon which collects these logs and writes them to a set of log entries. There is also an Audit dispatcher daemon (audispd) which provides a real-time mechanism for analytical programs to interact with.

Finally, there's a set of tools that can be used to configure event generation. These include auditctl, which allow configuration of settings and parameters of how events are generated, aureport, which can be used to generate a report of events collected by the audit daemon, and ausearch, which can be used to search through logs.

Installing auditd

In order to deploy auditd on Debian, you first ensure the audit daemon, and the audit dispatching framework are deployed on the system:

apt-get install auditd audispd-plugins

Deploying an initial configuration

Once these are installed, you can deploy an initial rule set by editing the file /etc/audit/rules.d/audit.rules. The following is a sensible rule set that gives you a few key elements. The following is a config I've successfully run, based on the Linux Auditd best practice configuration by Florian Roth, and the GDS config of the UK Government Digital Service. Thanks to them for developing and sharing some useful auditd starting point configurations.

## Auditd configuration
## Configuration hint:
## -w watch file system object, -p sets [r|w|x|a]
## -a add rule: exit, upon syscall exit

## Remove any existing rules

## Having a large buffer ensures we avoid dropping logs
-b 8192

## Failure Mode
## Possible values are 0 (silent), 1 (printk, print a failure message),
## and 2 (panic, halt the system).
-f 1

## Audit the audit logs, and execution of auditd reporting tools

-w /var/log/audit/ -k auditlog
-w /etc/audit/ -p wa -k auditconfig
-w /etc/libaudit.conf -p wa -k auditconfig
-w /etc/audisp/ -p wa -k audispconfig
-w /sbin/auditctl -p x -k audittools
-w /sbin/auditd -p x -k audittools
-a always,exit -F dir=/var/log/audit/ -F perm=r -F auid>=1000 -F auid!=unset -k audittools
-a always,exit -F path=/usr/sbin/ausearch -F perm=x -k audittools
-a always,exit -F path=/usr/sbin/aureport -F perm=x -k audittools
-a always,exit -F path=/usr/sbin/aulast -F perm=x -k audittools
-a always,exit -F path=/usr/sbin/aulastlogin -F perm=x -k audittools
-a always,exit -F path=/usr/sbin/auvirt -F perm=x -k audittools

## Log all process execution: disabled here by default due to load
# -a exit,always -S execve -k cmd
## Log all process executions by root
-a exit,always -F arch=b64 -F euid=0 -S execve -k rootcmd
-a exit,always -F arch=b32 -F euid=0 -S execve -k rootcmd

## Identify creation of filesystem nodes and file system mounts
-a exit,always -F arch=b32 -S mknod -S mknodat -k specialfiles
-a exit,always -F arch=b64 -S mknod -S mknodat -k specialfiles
-a exit,always -F arch=b32 -S mount -S umount -S umount2 -k mount
-a exit,always -F arch=b64 -S mount -S umount2 -k mount

## Clock and time zone changes
-a exit,always -F arch=b32 -S adjtimex -S settimeofday -S clock_settime -k time
-a exit,always -F arch=b64 -S adjtimex -S settimeofday -S clock_settime -k time
-w /etc/localtime -p wa -k localtime

## Updates to cron
-w /etc/cron.allow -p wa -k cron
-w /etc/cron.deny -p wa -k cron
-w /etc/cron.d/ -p wa -k cron
-w /etc/cron.daily/ -p wa -k cron
-w /etc/cron.hourly/ -p wa -k cron
-w /etc/cron.monthly/ -p wa -k cron
-w /etc/cron.weekly/ -p wa -k cron
-w /etc/crontab -p wa -k cron
-w /var/spool/cron/crontabs/ -k cron

## Credential/user/group/login changes
-w /etc/group -p wa -k etcgroup
-w /etc/passwd -p wa -k etcpasswd
-w /etc/gshadow -k etcgroup
-w /etc/shadow -k etcpasswd
-w /etc/security/opasswd -k opasswd
-w /usr/bin/passwd -p x -k passwd_modification
-w /usr/sbin/groupadd -p x -k group_modification
-w /usr/sbin/groupmod -p x -k group_modification
-w /usr/sbin/addgroup -p x -k group_modification
-w /usr/sbin/useradd -p x -k user_modification
-w /usr/sbin/usermod -p x -k user_modification
-w /usr/sbin/adduser -p x -k user_modification
-w /etc/login.defs -p wa -k login
-w /etc/securetty -p wa -k login
-w /var/log/faillog -p wa -k login
-w /var/log/lastlog -p wa -k login
-w /var/log/tallylog -p wa -k login

## Changes to network configurations
-w /etc/hosts -p wa -k hosts
-w /etc/network/ -p wa -k network

## system startup scripts
-w /etc/inittab -p wa -k init
-w /etc/init.d/ -p wa -k init
-w /etc/init/ -p wa -k init

## library search paths
-w /etc/ld.so.conf -p wa -k libpath

## kernel parameters
-w /etc/sysctl.conf -p wa -k sysctl

## modprobe configuration
-w /etc/modprobe.conf -p wa -k modprobe

## pam configuration
-w /etc/pam.d/ -p wa -k pam
-w /etc/security/limits.conf -p wa -k pam
-w /etc/security/pam_env.conf -p wa -k pam
-w /etc/security/namespace.conf -p wa -k pam
-w /etc/security/namespace.init -p wa -k pam

## system configuration changes
-w /etc/ssh/sshd_config -k sshd
-a exit,always -F arch=b32 -S sethostname -k hostname
-a exit,always -F arch=b64 -S sethostname -k hostname
-w /etc/issue -p wa -k etcissue
-w /etc/issue.net -p wa -k etcissue

## Capture all failures to access on critical elements
-a exit,always -F arch=b64 -S open -F dir=/etc -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/bin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/sbin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/usr/bin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/usr/sbin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/var -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/home -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/srv -F success=0 -k unauthedfileacess

## Monitor for use of process ID change (switching accounts) applications
-w /bin/su -p x -k priv_esc
-w /usr/bin/sudo -p x -k priv_esc
-w /etc/sudoers -p rw -k priv_esc

## Monitor usage of commands to change power state
-w /sbin/shutdown -p x -k power
-w /sbin/poweroff -p x -k power
-w /sbin/reboot -p x -k power
-w /sbin/halt -p x -k power

## Do not allow configuration changes
## -e 0 disables, -e 1 enables, -e 2 locks configuration until reboot
-e 1

Once the configuration is saved, restart auditd with /etc/init.d/auditd restart to reload the configuration.

Accessing logs

There are a number of ways to obtain logs and take action on them. First, logs are written to /var/log/audit.log on most systems. These logs look like the following:

type=SYSCALL msg=audit(1520912790.481:50015): arch=c000003e syscall=59 success=yes exit=0 a0=132ed88 a1=132ef48 a2=12de208 a3=5d3 items=2 ppid=18673 pid=18677 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1669 comm="tail" exe="/usr/bin/tail" key="rootcmd"
type=EXECVE msg=audit(1520912790.481:50015): argc=3 a0="tail" a1="-f" a2="audit.log"
type=CWD msg=audit(1520912790.481:50015): cwd="/var/log/audit"

These three logs were generated by one root command execution of tail -f /var/log/audit.log, using the above configuration. They show that in the current working directory (CWD) of /var/log/audit, the user root executed "tail -f audit.log".

Another way of accessing logs is through the use of aureport. Aureport, run without parameters, will generate an output showing the amount of events generated, and their distribution:

user@daemon-box:/var/log/audit$ sudo aureport

Summary Report
Range of time in logs: 02/20/2018 01:17:01.139 - 03/13/2018 03:48:03.960
Selected time for report: 02/20/2018 01:17:01 - 03/13/2018 03:48:03.960
Number of changes in configuration: 204
Number of changes to accounts, groups, or roles: 0
Number of logins: 9
Number of failed logins: 9
Number of authentications: 17
Number of failed authentications: 3
Number of users: 3
Number of terminals: 9
Number of host names: 3
Number of executables: 142
Number of commands: 350
Number of files: 595
Number of AVC's: 0
Number of MAC events: 9
Number of failed syscalls: 11422
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 0
Number of integrity events: 0
Number of virt events: 0
Number of keys: 33
Number of process IDs: 19394
Number of events: 42861

You can limit this report to a specific timeframe through:

aureport --start 03/01/2018 00:00:00 --end 03/30/2018 00:00:00

With ausearch, you can also get all logs related to a specific user:

ausearch -ua username

If you're interested in something a little bit more parseable, you may want to look into Slack's go-audit alternative to the Auditd daemon, which outputs logs in json.

Processing logs

You'll likely want to move auditd logs off of the machine promptly, to ensure they cannot be modified by an attacker (even if you are tracking access to audit trails using the sample configuration above). Use rsyslog, or another daemon that allows for transmission over TLS, or look into using audisp-remote, to log to a remote logging server.

Logs can be useful for a number of reasons, two of which stand out:

There's some interesting information on how to leverage auditd logs in Splunk here, and in Graylog here.


An alternative to Auditd you may want to look into is the Capsule8 sensor.