Parsing Lynis Audit Reports

Last week we passed along some information on a Unix audit tool called Lynis, maintained by Michael Boelen (https://cisofy.com/lynis/). The value of this tool is that it is an open script that auditors can give to system administrators to run on their Unix servers in order to assess specific technical security controls on the system.

If an auditor chose to use this tool to gather data, likely the process would look something like this:

  1. The auditor gives a copy of the script to the data custodian (system administrator).
  2. The system administrator runs the script on the target machine (note, it does not work across the network).
  3. Lynis produces an output file: /var/log/lynis-report.dat (location can be customized).
  4. The system administrator gives the auditor a copy of this output file.
  5. The auditor parses the file for relevant information to include in their findings report.

Unfortunately the output report isn’t quite as pretty as you might hope. The auditor will need to plan on spending some time parsing the outputs of the report file in order to gather useful information that can be included in their audit report. The data can most certainly be copy and pasted and manually edited, but why not use a little command line scripting to make our lives easier.

At a minimum, an auditor will want to parse a list of all the warnings and suggestions automatically made by the tool. Let’s use built in commands, like the Linux cat, grep, and sed commands. A simple command line command that will give an auditor the ability to parse out only the warnings made in the report file is the following:

cat /var/log/lynis-report.dat | grep warning | sed –e ‘s/warning\[\]\=//g’

The same can be done with the suggestions as well using the following command line syntax:

cat /var/log/lynis-report.dat | grep suggestion | sed –e ‘s/suggestion\[\]\=//g’

A couple other commands that you might want to play with to parse this file are the following commands as well. The following gathers a list of all installed software packages on the system:

cat /var/log/lynis-report.dat | grep installed_package | sed –e ‘s/installed_package\[\]\=//g’

The following command gathers a list of all the installed shells on the system from the report:

cat /var/log/lynis-report.dat | grep available_shell | sed –e ‘s/available_shell\[\]\=//g’

You get the idea. The system administrator can provide the auditor one output file, and the auditor could easily write a parsing script based on this syntax to completely parse the file into a manageable output that is a little more friendly to read.

We hope this inspires you to continue writing your own scripts to automate the audit process. The more we can automate, the faster our audit analysis. The faster our audit analysis, the more technical audits we can perform and the more likely our systems will be properly secured. Until next time…

Unix Auditing with Lynis

One of the questions I get asked often times in our audit classes is how to automate data collection from systems in a way that system administrators will trust. The problem is that there are a number of tools available for doing data collection, but often times those tools are compiled with no easy way to do code review on the tool. And rightfully so, sysadmins are normally cautious about installing software on their systems that they’ve not had the time to verify.

One solution to this is to write your own scripts for automated data collection. We tend to recommend keeping things simple with languages like PowerShell and Bash. One of the nice things about using these languages is that it allows the sysadmin the ability to do code review prior to running the tools. This can give them a little more comfort with the tools typically. But we don’t always have the time to write our own. So what then?

One option, at least for Unix, is to run a tool called Lynis (https://cisofy.com/lynis/). Michael Boelen is lead developer on this project, and from the site he says that:

“Lynis is an auditing tool for Unix (specialists). It scans the system and available software, to detect security issues. Beside security related information it will also scan for general system information, installed packages and configuration mistakes.

This software aims in assisting automated auditing, software patch management, vulnerability and malware scanning of Unix based systems. It can be run without prior installation, so inclusion on read only storage is no problem (USB stick, cd/dvd).

Lynis assists auditors in performing Basel II, GLBA, HIPAA, PCI DSS and SOX (Sarbanes-Oxley) compliance audits.

Intended audience:
Security specialists, penetration testers, system auditors, system/network managers.

Examples of audit tests:
– Available authentication methods
– Expired SSL certificates
– Outdated software
– User accounts without password
– Incorrect file permissions
– Firewall auditing”

In addition it has been found to run on the following Unix operating system platforms:

Arch Linux; CentOS; Debian; Fedora Core 4 and higher; FreeBSD; Gentoo; Knoppix; Mac OS X; Mandriva 2007; OpenBSD 4.x; OpenSolaris; OpenSuSE; PcBSD; PCLinuxOS; Red Hat, RHEL 5.x; Slackware 12.1; Solaris 10; Ubuntu

The nice thing about this tool is that even for junior Unix auditors, or senior auditors without a lot of experience on Unix systems, it is simple to gather information and potential findings on a system. Like similar tools on Microsoft Windows (think MBSA), Lynis gives auditors the ability to identify findings on a system simply by running the tool. There is no install necessary, simply unzip the tools and run the script.

The tool is free and the code is easy enough to review if you enjoy that kind of thing. Hopefully you will find that this becomes a staple of your audit toolkit. Enjoy!

Baselining as a Primary Audit Tool

So if I was trapped on a desert island and only had one audit tool that I could have with me to audit the island’s DHARMA systems, which would I want…

For me the answer would have to be baselines. As an auditor, ideally I want to ensure that an organization’s technology systems reflect a conscious decision on the part of the organization. In other words, I want to ensure that technology has been implemented, with a full understanding of the potential risks those systems pose, and that they have been implemented in a systematic, tested, and documented fashion. Or said another way, I don’t want to see an organization deploy systems in an ad hoc manner that exposes them to risk – I want to see controlled implementations.

A huge indicator of a controlled environment is documentation on the system being implemented. Specifically I’m looking for documented system baselines which demonstrates evidence of conscious decisions on the part of the organization to protect and secure their information.

RSA, a security division of EMC, in their Information Security Glossary defines baseline or baselining as the following:

“An effective method for identifying security attacks on a network, baselining starts by measuring normal activity on a network or network device. That measurement is used as threshold, or baseline, to detect unusual patterns or changes in levels of activity. With this method, the security expert can focus efforts on evaluating anomalies instead of looking for them by reviewing huge log files. The term is also used to refer to other security practices. A baseline, or security baseline often refers to an organizational standard for securely configuring network devices. It can also refer to the results of an organization’s first security assessment. This becomes the baseline against which the organization measures improvements and changes.”

For most auditors understanding the concept of a baseline is the easy part. The devil, it turns out, is in the details. The information auditors really want to know is what information should be baselined and then practically how do you go about performing that baseline. Of course the ultimate conclusion to this discussion would be to consider how to integrate baselines with an organization’s efforts for continuous monitoring and automation.

This year one of my goals is to help auditors by providing them resources which will enable them to more efficiently create baselines of their systems and later automate checks of those baselines. To that end I’ve decided this year I will be releasing a number of scripts to help auditors perform baselines of their systems. The scripts that I will be releasing this year will all be Microsoft PowerShell scripts, but I may throw in a few Unix Bash scripts along the way just for some diversity. So enjoy the scripts and feel free to make requests. Most importantly we want to be able to offer resources that will help you in your efforts to better secure your systems.

Wishing you a Happy New Year and a Secure 2011!